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Each  student  received  up  to  eight  hours  of  instruction  in  the  simulator  (mean  training  time:  6.57  hours)  with 
instruction  by  SAC  KC- 135  IPs  scheduled  over  a 2-day  period.  Following  simulator  training,  each  student  flew  two 
evaluation  sorties  in  the  KC-135  aircraft  with  three  to  four  landings  attempted  on  each  sortie. 

Simulator  Results.  All  subjects  demonstrated  an  increase  in  flying  skill  during  simulator  training.  Comparison 
of  group  mean  times  in  the  simulator  required  to  meet  a specified  proficiency  criterion  did  not  reveal  any 
differential  efficiency  (time  required  to  meet  proficiency)  among  the  three  systems. 

Evaluation  Flight  Results'.  A comparison  of  evaluation  flight  scores  revealed  that  the  Night  and  Day  systems 
trained  more  effect!  ely  than  the  TV  system.  The  TV  system  trained  less  effectively  in  the  final  approach  glidepath 
segment  of  the  landing  task.  No  serious/deficiencies  were  identified  in  the  Day  and  Night  system  which  might 
adversely  affect  the  quality  of  training. 

Follow  on  CCTS  Training  Results:  Sixty  percent  of  the  simulator  trained  students  received  the  "Highly 
Qualified"  score  on  their  checkride  landings,  while  only  thirty  percent  of  CCTS  students  normally  receive  this  score. 
Instructor  pilots  reported  a generally  higher  skill  level  from  the  students  who  had  received  simulator  training  in  this 
study. 
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PREFACE 


This  effort  was  conducted  by  the  Flying  Training  Division,  Air  Force  Human  Resources  laboratory, 
Williams  AFB,  Arizona.  The  research  was  completed  under  project  1123,  USAF  Flying  Training 
Development,  Mr.  James  F.  Smith,  Project  Scientist,  and  task  112303,  The  Exploration  of  Simulation  in 
Flight  Training,  Mr  Robert  Woodruff,  Task  Scientist. 

This  study  was  conducted  in  response  to  an  August  1976  statement  of  technical  need  from  the 
Simulator  Systems  Program  Office,  Aeronautical  Systems  Division  (ASD/SD24),  Wright-Patterson  AFB, 
Ohio,  and  the  Strategic  Air  Command  (HQ  SAC/XPHS),  Offutt  AFB,  Nebraska.  Because  of  the  mutual 
interests  in  the  study,  a joint  planning  group  was  organized  to  oversee  the  design,  planning,  and 
management  of  the  study  and  its  required  resources.  The  three  members  of  this  group  were  Capts  Charles 
Mote  (ASD/SD24),  Gene  Englund  (HQ  SAC/XPHS),  and  Jack  Thorpe  (AFHRL/FT). 

The  desire  to  impact  pending  simulator  hardware  procurements  coupled  with  the  magnitude  of  the 
study  forced  an  accelerated  research  schedule  and  increased  the  demands  upon  a number  of  people  directly 
or  indirectly  associated  with  the  execution  of  the  study.  It  was  exclusively  due  to  their  extreme  dedication 
and  outstanding  performance  under  these  severe  time  constraints  that  the  study  was  successfully  completed 
and  the  results  made  available  in  time  to  influence  decisions  regarding  significant  hardware  expenditures. 
Four  field  research  teams  carried  the  burden  of  this  effort. 

Day  Team:  (Site:  Boeing  Aerospace  Company,  Seattle,  Washington) 

On-Site  Experimenter:  Mr.  Dean  LeMaster 
Associate  Experimenter:  Airman  Lois  Short 
SAC  Instructor  Pilots: 

Maj  Harlan  Tate,  92  ARS,  Fairchild  AFB,  Washington  (Team  Chief) 

Capt  Wally  X.  Pickard,  905  ARS,  Grand  Forks  AFB,  North  Dakota 

Capt  Jim  Williams,  9 ARS,  Beale  AFB,  California 

Capt  Gary  L.  Curtis,  349  AREFS,  Beale  AFB,  California 

Wight  Team:  (Site:  American  Airlines  Right  Academy,  Ft.  Worth,  Texas) 

On-Site  Experimenter:  Capt  Bob  McFadden 
Associate  Experimenter:  Capt  Nick  Varney 
SAC  Instructor  Pilots: 

Capt  Jim  Mohan,  19  BMW,  Robins  AFB,  Georgia  (Team  Chief) 

Capt  Ron  Gordon,  91 1 ARS,  Seymour-Johnson  AFB,  North  Carolina 
Capt  Rhett  Cooper,  301  ARW,  Rickenbacker  AFB,  Ohio 

TV  Team:  (Sue:  American  Airlines  Right  Academy,  Ft.  Worth,  Texas) 

On-Site  Experimenter:  Capt  Nick  Varney 
Associate  Experimenter:  Dr.  Delores  Tyler 
SAC  Instructor  Pilots: 

Capt  Jon  Benner,  913  ARS,  Bardsdale  AFB,  Louisiana  (Team  Chief) 

Capt  Jerry  Moynihan,  7 ARS.  Carswell  AFB,  Texas 
Capt  Rod  Wells.  384  AREFS,  McConnell  AFB,  Kansas 

In-Flight  Evaluation  Team:  (Site:  Carswell  AFB,  Texas) 

Aircraft  Commander:  Capt  William  Poston,  93  BMW,  Castle  AFB,  California 
Jump  Scat  Observer:  Capt  John  Clark,  7 BMW,  Carswell  AFB,  Texas 
On-Board  Experimenter:  Capt  McFadden/Capt  Thorpe/Dr.  Tyler 

Runway  Video  Camera  Location:  Mr.  Bill  Brubaker,  Singer-Link  Company  (Williams  AFB, 
Arizona) 

Ground  Experimenter:  Capt  Varney 


The  exceptional  performance  of  the  twelve  Strategic  Air  Command  instructor  pilots  making  up  the 
four  field  teams  is  worthy  of  a special  commendation.  Each  of  these  pilots  demonstrated  exceptional 
talents  ^s  members  of  a scientific  research  team,  adopting  the  strict  rules  of  the  scientist  and  striving  to 


1 


achieve  the  highest  level  of  experimental  control.  Each  demonstrated  extraordinary  instructor  skills  in  the 
simulator  and  keenly  perceptive  evaluative  skills  in  the  aircraft.  Throughout  the  exhaustive  experimental 
schedule,  the  constant  contributions  of  these  research  pilots  continued  to  impress  everyone  associated  with 
the  project. 

Also  worthy  of  commendation  is  the  exceptional  support  from  the  two  contractor  teams  which 
provided  the  simulator  hardware.  At  the  American  Airlines  Right  Academy,  Mr.  Jerry  Wynn  directed  the 
effort  with  support  from  the  Simulator  Operational  Support  Branch,  especially  Mr.  Rodney  Botts.  Expert 
evaluations  of  student  performance  were  made  by  Right  Capts  Pete  Singleton  and  BiD  Marpee,  and  valuable 
assistance  and  insight  into  the  training  problem  was  received  from  Capts  William  Cooper  and  Walter 
Eskridge.  At  the  Boeing  Aerospace  Company,  Mr.  Robert  Trask  coordinated  the  effort,  relying  on  the 
outstanding  performance  of  the  Simulator  Engineering  Branch  under  the  supervision  of  Mr.  Ben  Warner  and 
Mr.  Bill  Kent.  Capt  Pat  Patterson  provided  excellent  evaluations  of  student  performance  and  Mr.  Gale 
Rhodes  accomplished  a key  technical  task  by  recording  simulator  parameters  onto  video  tape  during 
student  test  runs.  A special  thanks  is  due  Dr.  Conrad  Kraft  for  this  helpful  comments  and  critiques 
concerning  the  organization  and  conduct  of  the  experiment  itself. 

Mr.  Bill  Brubaker,  Singer-Link  Company,  Williams  AFB,  Arizona,  overcame  a number  of  technical 
obstacles  in  video  tape  recording  student  landings  from  an  isolated  runway  position.  He  was  able  to 
accomplish  this  task  only  because  of  the  capable  assistance  of  Maj  Joe  Monaghan  and  Capt  Jack  Massey, 
4235  (SAC),  Carswell  AFB,  Texas,  who  provided  equipment  as  well  as  expertise. 

A number  of  other  individuals  made  significant  contributions  to  the  project:  Capt  Bruce  Knapp 
(8AF/DOT),  LtCol  Jan  DeMuth  (HQ  SAC/XPHS),  LtCol  Steve  Mussleman  (HQ  SAC/DOTP),  Capt  Frank 
Ramsey  (15AF/DOT),  Majs  Jerry  Rhynne  and  Ken  Cromwell  (93  BMW/DOT),  and  TSgt  Ted  Schmalbeck 
(AFHRL/FT).  Significant  technical  assistance  was  received  from  Mr.  Michael  Cyrus  (AFHRL/FT).  Capt 
Barry  McFarland  (ASD/ENF.C),  and  Mr.  Dan  Kugel  (ASD/SD24). 

The  heavy  administrative  workload  associated  with  the  conduct  of  the  study  and  the  production  of 
this  draft  report  was  borne  by  Ms.  Joann  Hill  and  Ms.  Marji  Scotten  (AFHRL/FT).  The  AFHRL/FT 
Research  Review  Committee  made  a number  of  valuable  suggestions  during  earlier  drafts  of  the  report 
which  improved  its  clarity  and  organization.  Dr.  Wayne  Waag  (AFHRL/FT)  supervised  the  final  review  and 
assembly  of  the  report,  and  also  contributed  significantly  to  the  statistical  data  analyses. 

Special  thanks  goes  to  our  branch  chief  Mr.  Jim  Smith  for  his  staunch  support  of  a project  which 
drained  a large  portion  of  his  scientific  manpower  resources  for  several  months. 
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TRAINING  EFFECTIVENESS  OF  THREE  TYPES  OF  VISUAL  SYSTEMS 
FOR  KC-135  FLIGHT  SIMULATORS 
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I.  INTRODUCTION 

The  technology  of  flight  simulator  visual  systems  is  advancing  rapidly.  Variety  and  quality  have 
improved,  making  procurement  decisions  increasingly  difficult.  Costs  must  be  considered -not  only  for  the 
initial  acquisition,  but  for  projected  operation  and  maintenance  as  well.  Likewise,  the  training  effectiveness 
of  the  system  must  be  considered,  that  is,  the  ability  of  the  system  to  effectively  accomplish  the  training 
mission  required  by  the  user.  The  present  study  was  designed  to  provide  training  effectiveness  information 
for  one  specific  application-three  generic  types  of  visual  systems  being  considered  for  KC-135  Combat 
Crew  Training  School  (CCTS).  One  of  these  systems  will  be  used  for  transition  training  of  student  copilots 
and  upgrading  of  aircraft  commanders  in  the  KC-135  tanker.  The  three  candidate  systems  considered  in  this 
study  were  the  Day/Night  Color  Computer  Generated  Image  (CGI),  Night  Only/Point  Light  Source  CGI, 
and  TV/Model  Board.  For  convenience,  they  are  designated  Day,  Ni$it  and  TV. 

Each  of  these  systems  has  been  used  successfully  by  commercial  airlines  to  train  their  pilots.  Whether 
each  system  is  equally  effective  for  Air  Force  training  was  unknown.  For  one,  the  experience  level  of  the 
learner  is  different.  The  Air  Force  KC-135  student  copilot  is  a considerably  less  experienced,  younger  pilot 
than  his  airline  counterpart  upgrading  into  a Boeing  707.  In  addition,  some  Air  Force  piloting  skills  are 
unique,  such  as  in-flight  refueling  and  the  flight  duties  of  a crew  during  an  alert. 

The  present  study  examined  each  of  the  three  visual  systems  to  see  if  they  would  be  effective  for  use 
in  KC-135  CCTS.  This  examination  took  the  form  of  a structured  and  controlled  experiment  comparing  the 
training  effectiveness  of  the  three  systems  for  a given  class  of  pilots  on  a given  task. 


II.  METHOD 


General  Approach 

The  experiment  used  a transfer  of  training  design.  Subjects  were  trained  on  a given  visual  system  in 
the  simulator  and  then  evaluated  in  the  aircraft.  The  tmount  of  skill  carry  over,  or  transfer,  was 
operationally  defined  as  the  training  effectiveness  of  each  visual  system.  A comparison  of  the  effective 
transfer  of  each  system  indicates  which  system  is  best  for  a given  subject  population  on  a given  task.  In  this 
experiment,  three  groups  of  ten  pilots  participated,  each  group  being  trained  using  one  visual  system  before 
proceeding  to  the  aircraft  for  evaluation. 

Since  it  was  beyond  the  scope  of  this  study  to  systematically  evaluate  each  visual  system  across  all  of 
the  training  conditions  for  which  it  will  probably  be  used  at  CCTS,  a research  strategy  was  employed  which 
selected  the  most  critical  and  demanding  subject  and  task  variables  in  order  to  assess  the  maximum 
capabilities  of  each  visual  system.  The  subjects  were  recent  graduates  of  USAF  Undergraduate  Pilot 
Training  (UPT)  who  were  transitioning  into  the  copilot  crew  position  in  the  KC-135.  All  subjects  had  been 
scheduled  for  CCTS  but  had  not  started  training.  None  had  previous  heavy  aircraft  or  multiengine  flying 
experience  nor  did  any  have  academic  or  flight  training  in  the  KC-135.  It  was  expected  that  a less 
experienced  pilot  would  demonstrate  the  most  skill  acquisition  during  training,  thus  highlighting  any 
relative  differences  in  training  effectiveness  of  the  different  viiual  systems. 

The  training  task  was  the  visual  traffic  pattern,  approach,  and  landing.  This  is  one  of  the  most  visually 
dependent  tasks  executed  in  the  KC-135.  It  is  also  one  of  the  most  difficult  tasks  performed  in  the  aircraft. 
It  was  expected  that  this  task  would  exercise  each  visual  system  to  the  maximum  extent,  again  accenting 
the  training  effectiveness  of  each  system. 


It  is  important  to  note  that  this  study  was  a hardware  evaluation.  Instructional  strategies  and  training 
techniques  were  not  investigated.  Several  important  components  of  the  overall  training  scheme  were  held 
constant.  The  simulator  syllabus  was  standardized  for  all  subjects  and  closely  followed.  The  instructional 
techniques  available  to  the  simulator  instructor  pilot  (IP)  were  precisely  defined  and  monitored.  The  visual 
environments  presented  by  each  visual  system  were  not  changed  to  optimize  training  for  the  particular  pilot 
experience  level  or  task  being  studied.  The  visual  environments  which  were  used  were  those  which  had  been 
created  by  the  contractor  or  manufacturer  to  be  the  most  faithful  reproduction  of  the  airdromes  specitied. 

Subjects 

Thirty  pilots  transitioning  into  the  copilot  crew  position  in  the  KC-135  aircraft  participated  as 
subjects  in  this  study.  Each  pilot  was  a recent  graduate  of  UPT  and  had  been  scheduled  to  attend  KC-135 
CCTS,  but  had  not  started  training.  The  average  age  was  25  years,  with  the  range  extending  from  23  years 
to  28  years.  No  pilot  had  previous  flying  experience  in  a large  multiengine  aircraft.  However,  some  had 
piloting  time  in  addition  to  UPT:  seven  pilots  had  logged  over  100  hours  in  light  aircraft  with  the  mo6t 
experience  being  700,  725,  and  1,200  hours;  two  other  pilots  had  helicopter  time  from  prior  service  in  the 
Army  (1,800  hours  and  2,100  hours);  a third  pilot  had  60  hours  in  sailplanes; a fourth  had  25  hours  in  an 
F4  fighter. 

The  30  subjects  came  from  two  CCTS  classes  of  15  pilots  each.  The  first  class  (K77-008,  Training 
Session  I)  participated  in  early  January  1977.  The  second  class  (K77-009,  Training  Session  II)  participated 
in  early  February  1977.  Each  class  was  divided  into  three  groups  of  five  subjects  each,  and  each  group  was 
assigned  to  one  of  the  three  visual  systems,  making  a total  of  10  subjects  per  visual  system  for  the  overall 
experiment.  The  10  subjects  trained  on  the  Day  system  are  referred  to  as  the  Day  Group.  Similarly,  the 
Night  and  TV  Groups  were  trained  on  the  Night  and  TV  systems,  respectively.  Individual  subjects  within 
groups  are  designated  as  D1  through  D10,  N1  through  N10,  or  TV1  through  TV10. 

For  administrative  purposes  all  subjects  reported  initially  to  CCTS,  Castle  AFB,  California,  before 
departing  to  their  respective  on-site  simulator  training  locations.  No  instruction  about  the  KC-135,  either 
academic  or  flight,  was  given  during  this  brief  period  at  Castle  AFB. 

Instructors 

Nine  SAC  KC-135  IPs  conducted  all  of  the  simulator  instruction.  They  were  formed  into  three  teams 
of  three  IPs  each,  one  team  for  each  visual  system,  with  two  members  of  the  team  being  designated  as 
principal  instructors  and  one  member  performing  the  duties  of  the  team  chief. 

The  instructional  strategy  developed  for  use  in  the  simulator  is  best  described  as  a team  approach.  For 
each  training  session,  each  IP  team  instructed  five  subjects.  1PI  instructed  the  fust  and  fourth,  IP2 
instructed  the  second  and  fifth,  and  the  team  chief  instructed  the  third.  Each  instructor  remained  with  his 
subject  for  the  total  training  session  and  was  the  primary  source  of  instruction  in  the  cockpit.  However, 
during  much  of  the  training,  the  team  chief  and  often  the  other  IP  sat  in  on  the  cockpit  sessions  and 
assisted  the  IP.  (When  the  team  chief  instructed,  either  IP1  or  IP2  assisted,  and  often  both  participated  in 
instruction.)  This  assistance  was  in  the  form  of  helpful  comments  to  the  subject,  an  assessment  of  the 
subject’s  major  problem  areas,  and  suggestions  of  different  instructional  approaches  which  might  help  solve 
these  problems.  The  team  members  met  frequently  in  conference  with  the  research  staff  to  analyze  subject 
weaknesses  and  devise  strategies  for  strengthening  these  weak  areas.  In  accordance  with  the  experimental 
procedures,  the  research  staff  monitored  the  progress  of  training  and  insured  that  all  data  were  collected 
and  recorded. 

Simulators 

In  this  report  “simulator”  is  used  to  mean  a simulator  equipped  with  one  of  the  three  visual  systems. 
Training  was  accomplished  in  Boeing  707  aircraft  simulators  rented  from  commercial  flight  training  sources 
(the  Day  system  from  the  Boeing  Aerospace  Company,  Seattle,  Washington,  and  the  Night  and  TV  Systems 
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from  the  American  Airlines  Flight  Academy,  Ft.  Worth,  Texas).  Commercial  sources  were  used  because  of 
the  nonavailability  of  government-owned  KC-135  flight  simulators  equippped  with  the  visual  systems  of 
interest,  engineering  calculations  of  various  hardware  and  software  response  parameters  (lags  and 
interactions,  for  example)  were  not  available  and  an  attempt  to  collect  them  was  unsuccessful.  The 
simulators  were  used  in  the  same  configurations  and  with  the  same  calibrations  normally  used  by  the 
contractor  during  commercial  pilot  training,  with  the  same  fine  tuning  of  the  devices  by  each  company's 
senior  pilots  and  simulator  engineering  staff. 

The  configuration  of  the  visual  system  field-of-view  (FOV)  for  this  study  required  a forward  FOV  for 
the  mstructor  and  a forward  and  side  FOV  for  the  subject,  since  the  pending  procurement  would  have  side 
windows  for  the  student.  The  Day  and  Night  systems  were  available  with  side  windows  on  either  the  left  or 
right.  The  TV/Model  Board  system,  however,  was  only  available  with  a left  side  window . The  current 
technology  does  not  yet  allow  a single  television  probe  to  reproduce  the  forward,  as  well  as  the  side  FOV 
with  an  acceptable  picture.  Therefore,  two  model  boards  and  two  probes  were  used  for  the  study,  one 
piobe  facing  forward  and  one  probe  facing  to  the  side.  The  system  that  was  used  could  only  reproduce  the 
left  FOV.  Therefore,  the  Day,  Night,  and  TV  systems  were  standardized  for  left  FOV  presentation, 
requiring  left  seat  subject  training  and  left-hand  traffic  patterns.  Accordingly,  left-hand  traffic  patterns  were 
tlown  in  the  aircraft  to  evaluate  training  transfer  except  in  cases  where  conflicting  traffic  or  local  rules 
forced  right-hand  patterns.  Lach  system  was  installed  on  a device  which  simulated  the  commercial  airline 
version  of  the  Boeing  707  aircraft  equipped  with  high  thrust  fan-jet  engines.  Due  to  differences  between  the 
707  and  the  KC-135  tanker,  an  assessment  was  made  by  CCTS  and  Strategic  Air  Command  Headquarters 
personnel  on  the  possible  negative  transfer  effects  which  could  occur  from  training  on  a 707  and  flying  a 
KC-135.  The  major  recommendations  were  to  change  the  airspeed  indicator  in  the  707  simulator  (a 
mach/airspeed  combined  instrument)  to  a drum  pointer  indicator  of  the  type  used  in  the  KC-135  and  to 
replace  the  (light  director  in  the  simulator  with  the  Collins  FD  109  used  in  the  KC-135.  The  flight  director 
was  modified  as  recommended  using  a Collins  FD  105  (identical  to  the  FD  109  except  for  its  slightly 
smaller  size),  but  the  airspeed  indicator  modification  was  not  accomplished  due  to  the  nonavailability  of 
parts.  The  differences  between  the  simulator  and  aircraft  were  explained  to  the  subject  at  the  start  of 
simulator  training.  A similar  briefing  was  given  prior  to  the  first  aircraft  sortie,  emphasizing  differences 
which  could  affect  performance  if  not  observed, e.g.,  fuel  flow  indicator  calibrations,  which  could  cause  an 
incorrect  thrust  setting  if  not  read  properly. 

Bach  simulator  had  two  pilot  positions  (a  left  seat  and  right  seat),  a navigator/flight  engineer  station 
located  behind  the  right  pilot’s  seat,  an  observer’s  chair  behind  the  left  pilot’s  seat,  and  a simulator 
operator’s  console  at  the  rear  of  the  cabin.  The  console  served  as  the  experimenter  station  and  was  occupied 
during  all  simulator  periods  by  an  on-site  experimenter.  It  was  from  this  position  that  the  elapsed  times  of 
various  elements  of  the  training  were  kept,  instructor  and  subject  protocols  recorded,  the  execution  of  the 
syllabus  monitored,  and  the  overall  supervision  of  the  experiment  carried  out. 

Because  of  the  training  emphasis  on  flying  the  approach  and  landing  with  primarily  visual  references, 
the  cockpit  and  visual  environments  were  modified  in  the  following  ways:  The  subject's  instrument  landing 
system  (1LS)  was  turned  off  (the  IPs  1LS  remained  operating  to  assist  him  instruct  and  evaluate  subject 
performance);  the  subject’s  radio  altimeter  was  masked;  the  subject’s  ILS  marker  beacon  lights  (outer, 
middle,  and  inner  markers)  were  masked  and  the  IP’s  marker  lights  were  blocked  from  the  subject’s  view; 
and  visual  approach  slope  indicator  (VASI)  lights  were  taken  out  of  the  visual  model.  Approach  strobe 
lights  were  left  in  the  visual  model. 

The  visual  environment  used  during  simulator  training  was  different  for  each  visual  system , partly  due 
to  the  type  of  model  each  system  was  capable  of  creating  and  partly  due  to  the  types  of  models  available  in 
each  system.  Table  1 describes  each  of  the  models  used. 

Each  simulator  was  equipped  with  a 3 degree-of-freedom  motion  platform  for  cockpit  motion  cues. 
The  motion  platform  was  operational  for  all  training  periods. 
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Table  1.  Characteristics  of  Visual  Models 


Dey  System 

Night  System 

TV  System 

A Standard  “Anytown  USA" 

Airdrome  Modeled 

Grant  County  Airport, 
Moses  Lake,  Washington 

Dulles  International 
Airport 

After  Any  specific 
Airdrome 

Runway  Characteristics 

Runway  32R 

Standard  Category  11 

1 3,500'  X 300’ 

No  VASIS 

Approach  Strobes  on 

Runway  36R 

Standard  Category  II 

12,000’  X2001 

No  VASIS 

Approach  Strobes  on 

Runway  35 L 

Standard  Category  II 

1 1 ,b00'  X 200' 

No  VASIS 

Approach  Strobes  on 

Major  Ground  Cues 

Standard  Airdrome 

Features 

Large  Hangar  on  Lett  of 
Touchdown  Area 

Surrounding  Area  Made  of 
Large  Fields,  a Lake, 
and  Road  Lines 

Lights  and  Shading  Used 
to  Create  Airdrome 

Features 

Intersecting  Runway  With 
Moving  Traffic 

12,000'  Parallel  Runway 

Standard  Airdrome 

Features 

industrial  Area  2 Miles 

Off  Fnd  of  Runway 

5,000'  Parallel  Runway 
to  Right  of  35  L 

Accurately  Scaled  Trees 
Buildings,  and  Ground 

Texture 

Other 

Visibility  Set  at  35 NM 
to  Reduce  Raster  Detail 

No  altitude  Limit 

Northern  Hemispheric 
Constellations  Modeled 

No  Altitude  Limit 

Altitude  Restricted  to 

1 ,900'  ACL  and  Below 

Simulator  Training 

Upon  arrival  at  the  simulator  site,  subjects  were  given  a general  overview  of  the  study  and  the 
schedule  for  the  training.  Rules  for  proper  crew  rest  during  the  experiment  were  explained.  The  next  day 
was  used  for  subject  familiarization  at  the  simulator  building  and  for  crew  rest.  Tire  following  day  was  the 
start  of  the  2-day  simulator  training  session.  Prior  to  the  first  simulator  period  the  IP  prebrief  was  given, 
presenting  technical  information  required  to  fly  the  simulator  and  the  aircraft  (see  Appendix  A). 

Each  subject  was  scheduled  for  2 days  of  simulator  training  4 hours  of  instruction  each  day,  but  the 
subject  could  complete  the  simulator  training  in  less  time  if  he  met  certain  proficiency  criteria  prior  to  the 
end  of  the  eighth  hour.  These  proficiency  criteria  are  discussed  in  a later  section. 

Simulator  periods  were  scheduled  according  to  the  study  syllabus  prepared  for  this  experiment 
(Appendix  B):  a 1.0-hour  introductory  period  and  two  1.5-hour  periods  on  Day  1;  and,  two  1. 5-hour 
periods,  and  a final  1.0-hour  period  on  Day  2.  This  schedule  was  designed  so  that  the  three  daily  periods 
were  each  separated  by  a 1.0-  to  1.5-hour  rest  period.  Normally  each  subject  was  on-site  8 to  9 hours  per 
day. 

The  daily  scheduling  of  the  training  sessions  at  each  contractor  site  was  dependent  upon  device 
availability.  All  Night  training  (January  and  February)  and  the  January  Day  training  was  accomplished 
between  0800  and  1600  hours.  Some  of  the  February  Day  sessions  went  late  into  the  evening.  All  of  the 
TV  training  (January  and  February)  was  conducted  between  1600  and  2400  hours  since  the  full-time  use  of 
two  TV/Model  Boards  could  not  be  accommodated  during  normal  commercial  pilot  daytime  training 
periods. 

Syllabus.  The  major  element  in  the  syllabus  was  the  teaching  of  the  landing  task.  The  landing  task 
began  with  initialization  of  the  simulator  on  base  leg,  1,000'  above  ground  level  (AGL),  at  135  knots 
indicated  airspeed  (KIAS),  50 7c  flaps,  gear  down,  and  trimmed  for  straight-and-level  flight.  At  this  point, 
the  subject  assumed  control  and  flew  the  base  leg,  picked  a turn  point,  made  a left-hand  level  tum  onto 
final  approach,  maintained  level  flight  aligned  with  the  runway  centerline  until  intercepting  the  glide  path. 
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began  a descent,  followed  tne  glide  path  down  to  the  flare  point,  flared,  held  a proper  landing  attitude,  and 
touched  down.  The  IP  then  directed  the  student  to  apply  power  and  rotate  the  simulator.  The  simulator 
was  then  reinitialized  back  to  the  base  leg  starting  point.  At  this  point,  the  IP  normally  offered  critique  and 
instruction  while  the  simulator  was  stationary.  He  then  established  the  initial  base  leg  conditions  and  the 
subject  began  a new  landing  task.  Instruction  was  also  given  during  the  landing  task  itself.  Each  task  took 
approximately  3 minutes,  unless  position  freeze  was  used  to  halt  the  simulator  in  mid-approach  for 
instruction. 

Each  training  period  followed  the  syllabus  as  shown  in  Appendix  B.  The  first  period  was  flown 
without  winds  or  turbulence.  These  two  atmospheric  conditions  were  introduced  at  some  point  during  the 
second  period,  the  exact  point  being  at  the  discretion  of  the  IP.  For  the  remaining  periods  a wind  condition 
was  introduced  on  all  landings.  Three  wind  conditions  were  used  , a 15-knot  left  crosswind,  a 15-knot  right 
crosswind,  and  a 15-knot  headwind.  Each  condition  was  presented  with  the  same  approximate  frequency.  A 
random  chart  was  used  to  select  the  wind  condition  on  each  landing.  Wind  direction  was  announced  to  the 
subject  prior  to  the  beginning  of  each  landing  task. 

Each  subject  received  a maximum  of  8.0  hours  of  the  time  in  the  simulator,  of  which  7.25  hours  were 
used  for  training  and  the  final  .75  hour  for  evaluation.  Every  subject  received  the  final  evaluation  period, 
although  he  could  finish  training  in  less  than  7.25  hours  if  he  attained  proficiency  early. 

Proficiency.  A subject  was  considered  proficient  in  the  landing  task  when  he  was  able  to  accomplish 
five  successive  unaided  landing  tasks  within  the  parameters  shown  in  Table  2.  By  unaided,  it  was  meant 
without  verbal  or  physical  inputs  by  the  IP  or  team  chief  to  assist  the  subject  in  performing  the  landing. 
Momentary  excursions  outside  the  performance  standards  in  Table  2 were  allowed  at  the  discretion  of  the 
IP. 


Table  2.  Performance  Standards  for  an  Unaided  Landing* 


Landing  Task 

Element 

Para  malar 

Norm 

Accaptabla 

Tolaranca 

Base  Leg 

Altitude 

1 ,000'  AGL 

900'-!  ,100'  AGL 

Airspeed 

135K1AS 

130-145  K1AS 

Final  Turn 

Altitude 

1,000'  AGL 

900’-l,100'  AGL 

Airspeed 

135  KIAS 

130-145  KIAS 

Final  Approach 

Altitude 

1,000'  AGL 

900’-l,i00'  AGL 

Prior  to  Descent 

Airspeed 

135KIAS 

130-145  KIAS 

Final  Approach 

Airspeed 

135K1AS 

130-145  KIAS 

on  Glide  path 

Glide  path 

0 Deviation 

± 1 Dot  on 
Glidescope 

Ground  Track 
(Centerline 

Alignment) 

0 Deviation 

1 1 Dot  on 
Localizer 

Touchdown 

Bank  (At  Touchdown) 

0° 

±3° 

Touchdown  Point 

l,000'b 

0’  to  3,000'b 

Touchdown  Point 

Nosewheel  on 

Centerhalf  of 

(Lateral) 

Centerline 

Runway 

aAny  unsafe  control  input  or  aircraft  condition  was  considered  out  of  tolerance,  and  the  landing  was  not  counted 
toward  proficiency. 

bPast  runway  threshold. 
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As  the  subject  approached  proficiency  during  a training  period,  the  IP  normally  announced  an 
“in-cockpit  break”  and  he  and  the  team  chief  left  the  cockpit  for  a brief  conference  with  the  experimenter. 
If  the  IP  and  team  chief  agreed  that  the  subject  was  nearing  the  desired  proficiency  level,  they  returned  to 
the  cockpit  and  began  to  unobtrusively  count  successive  landing  tasks  which  were  completed  within  the 
allowable  tolerances.  No  instruction  was  given  during  the  landings,  although  IP  critique  and  instruction 
were  permitted  between  task  repetitions  while  the  simulator  was  being  reset.  If  a landing  was  completed 
within  the  tolerances  listed  in  Table  2,  the  IP  discretely  signaled  the  experimenter  who  recorded  the  event 
on  the  experiment  log.  If  five  successive  landings  were  successfully  completed,  the  IP  initiated  a “full  stop” 
following  the  touchdown  on  the  fifth  landing.  The  subject  was  declared  proficient  and  the  session  was 
terminated.  The  final  evaluation  was  scheduled  for  a later  time  and  the  subject  returned  at  that  time  to 
complete  the  final  .75-hour  period. 

Two  variations  to  this  procedure  could  occur.  If  there  was  insufficient  time  remaining  in  a given 
period  to  attempt  five  landings,  the  testing  for  proficiency  could  be  deferred  to  the  next  period.  Secondly, 
if  the  beginning  of  the  count  was  foDowed  by  a deterioration  in  the  student’s  performance,  the  IP  could 
elect  to  resume  his  instruction  during  the  landing  task  and  defer  the  count  to  a later  time. 

Following  completion  of  the  simulator  training  sessions  each  subject  traveled  from  the  simulator  site 
to  Carswell  AFB,  Texas,  for  the  aircraft  evaluation  portion  of  the  study. 

Evaluation  Flights 

Each  subject  flew  on  two  KC-135  aircraft  flights  scheduled  on  successive  days.  Five  subjects  were  on 
board  each  flight  which  lasted  approximately  3.0  to  3.5  hours.  For  example,  subjects  N1  through  N5  flew 
on  the  first  two  flights,  El  and  E2.  Each  subject  flew  a minimum  of  three  approaches  and  landings  per 
fligi’ t (except  as  noted  later). 

The  flight  schedule  was  designed  to  minimize  the  time  between  the  subject’s  final  simulator  period 
and  his  first  evaluation  flight,  given  a number  of  scheduling  constraints  such  as  CCTS  class  start  dates, 
weekend  flying  schedules,  and  aircraft  availability.  The  final  schedule  was  the  best  overall  reduction  of  idle 
days  for  the  maximum  number  of  subjects. 

Two  KC-135  senior  pilots  served  as  evaluators  on  all  12  aircraft  flights.  The  aircraft  commander  was 
from  the  CCTS  Standardization  and  Evaluation  Division  and  was  highly  experienced  in  evaluating  the 
performance  of  student  copilots.  The  other  evaluator  was  a senior  squadron  pilot  familiar  with  the 
Carswell  AFB  area.  He  occupied  the  jump  seat  during  the  aircraft  flights  and  was  designated  as  the  jump 
seat  observer  (JSO).  Also  on  board  were  IPs  from  the  simulator  teams:  one  IP  was  on  board  for  El , two  for 
E2,  and  three  for  E3  through  E12.  One  experimenter  flew  on  each  flight  and  at  least  two  experimenters  and 
one  IP  monitored  the  progress  of  the  flight  from  the  ground.  Prior  to  the  first  of  the  two  evaluation  flights, 
the  aircraft  commander  briefed  the  five  subjects  on  the  flight  profile,  how  he  would  give  control  of  the 
aircraft  to  each  of  them,  and  how  and  when  he  would  resume  control  of  the  aircraft.  He  briefed  the 
differences  between  the  707  simulator  and  the  aircraft  in  terms  of  thrust,  control  loading  and  response, 
lags,  aerodynamic  characteristics,  and  proper  flight  parameters  (e.g.,  airspeed,  altitude,  etc.). 

The  crew,  research  staff,  and  subjects  were  transported  to  the  aircraft  and  the  subjects  were  given  the 
appropriate  safety  briefings.  Each  subject  was  permitted  to  quickly  inspect  the  cockpit  upon  entering  the 
aircraft,  but  was  not  allowed  to  sit  in  either  pilot’s  seat  to  observe  cockpit  height  above  the  ground.  Each 
subject  remained  seated  in  the  rear  of  the  aircraft  for  the  takeoff  and  throughout  the  flight,  except  when  he 
flew  the  aircraft.  The  subject  first  entered  the  cockpit  during  the  crosswind  or  early  downwind  portion  of 
the  pattern,  took  his  position  in  the  left  seat  and  was  helped  with  his  harness  and  seat  adjustment  by  the 
JSO.  He  wore  a standard  headset  although  it  remained  disconnected.  Communication  between  aircraft 
commander  and  the  subject  was  accomplished  by  loud  talking.  Proceeding  downwind,  the  aircraft 
commander  permitted  the  subject  to  become  comfortable  and  observe  the  predominant  terrain  features  and 
location  of  the  runway.  He  then  let  the  subject  fly  the  remaining  portion  of  the  downwind  leg  to  get  the 
feel  of  the  controls.  The  aircraft  commander  took  the  plane  for  the  turn  to  base,  established  1.000’  AGL 


altitude  and  the  proper  airspeed  for  the  given  aircraft  weight  (marked  on  a white  card  on  the  console 
separating  the  two  pilots)  and  trimmed  the  aircraft  for  straight -and-level  flight.  He  then  gave  the  aircraft  to 
the  subject  and  pointed  out  the  runway.  The  subject  flew  the  final  approach,  flared,  and  landed  the  aircraft. 
He  then  advanced  the  throttles  under  the  direction  of  the  aircraft  commander  and  rotated  the  aircraft  at 
which  time  the  aircraft  commander  took  control. 

The  flight  crew  did  not  aid  the  subject  in  any  way  while  he  was  making  his  approach  and  landing.  No 
instruction  was  given.  The  aircraft  commander  did  make  control  inputs  when,  in  his  judgment,  the  aircraft 
was  getting  too  far  out  of  tolerance.  In  such  cases,  this  input  was  recorded  on  the  performance  data  card 
and  was  considered  in  the  subject’s  score.  For  all  flights,  the  subject’s  ILS  and  radio  altimeter  were  turned 
off. 

After  each  subject  flew  three  approaches,  the  next  subject  moved  into  the  left  seat.  If  sufficient  fuel 
remained  after  the  fifth  subject  had  completed  his  three  approaches,  subjects  were  rotated  back  into  the 
cockpit  one  by  one  to  fly  an  additional  landing. 

Except  for  the  approaches  made  on  El  1 and  a few  approaches  made  on  other  evaluation  flights,  all 
approaches  were  left-hand  patterns.  On  El  1 , flown  at  Blytheville  AFB,  Arkansas,  only  right-hand  patterns 
were  allowed  and  all  15  patterns  were  flown  with  right  turns  to  final. 

Following  their  second  flight,  subjects  were  debriefed  on  their  performance  and  permitted  to  ask 
questions  about  the  study.  The  flight  crew  discussed  the  genera]  performance  of  the  subjects  and  suggested 
ways  they  might  improve  their  performance.  The  subjects  then  returned  to  Castle  AFB  to  begin  CCTS 
academics. 

Evaluation  Flight  Location.  Table  3 shows  the  location  and  subject  composition  of  each  pair  of 
flights.  All  flights  were  initially  scheduled  to  be  flown  at  Carswell  AFB,  but  poor  weather  forced  El  and  E2 
to  be  flown  at  Barksdale  AFB,  near  Shreveport,  Louisiana.  The  remaining  flights  in  Training  Session  I 
(E3-E6)  were  flown  at  Carswell  AFB. 


Table  3. 

Evaluation  Flight  Location  and  Subject  Composition 

Location 

Flight 

(AFB) 

Subjects 

Training  Session  1 

El 

Barksdale 

NI.N2,  N3,  N4,  N5 

E2 

Barksdale 

E3 

Carswell 

TV1,  TV2,  TV3,  Dl,  D2 

E4 

Carswell 

E5 

Carswell 

TV4,  TV5,  D3,  D4.D5 

E6 

Carswell 

Training  Session  11 

E7 

Carswell 

N6,  N7,  N8,  D6,  D7 

E8 

Barksdale 

E9 

Carswell 

N9,  N10,  D8,  D9,  DIO 

E10 

Carswell 

Ell 

Blytheville 

TV6,  TV7,  TV8,  TV9,  TV10 

E12 

Carswell 

As  seen  in  Table  3,  all  five  E1-E2  subjects  were  from  the  Night  group.  Their  performance  appeared 
to  be  outstanding  when  compared  with  the  performance  of  Day  and  TV  group  subjects  on  E3-E6.  The 
question  that  immediately  surfaced  was  whether  their  performance  was  in  some  way  attributable  to  the 
Barksdale  flying  environment,  rather  than  to  their  particular  visual  system  training.  The  runway  at 
Barksdale  is  highlighted  by  flat,  undeveloped,  uncluttered  terrain  of  homogeneous  colors.  Carswell,  on  the 
other  hand,  is  approached  over  a populated,  hilly  area  with  many  buildings  surrounding  the  base  and  a large 
plant  facility  on  one  side  of  the  runway.  The  flight  crew  and  research  staff  considered  the  Carswell  runway 
more  difficult  to  see  than  the  Barksdale  runway.  In  addition,  the  Barksdale  VASI  lights  remained  on  during 
the  El  and  E2  landings  in  spite  of  attempts  to  get  them  turned  off.  These  lights  provide  visual  glideslope 
information  by  indicating  high  or  low  glideslope  attitudes  by  their  color.  Since  it  was  not  known  whether 
the  good  performance  of  the  Night  group  on  El— E2  was  attributable  in  part  to  the  Barksdale  runway 
environment  and  VASI  lights,  two  flights  in  Training  Session  II  with  Day  and  TV  group  subjects  were 
scheduled  to  return  to  Barksdale  (E8  and  El  1 ) to  try  to  make  this  determination.  Weather  forced  Ell  to  be 
moved  further  east  to  Blytheville  AFB,  which  is  similar  to  Barksdale ;i.e.,  the  terrain  is  flat  and  the  runway 
easily  detectable.  VASI  lights  remained  on  for  ES  and  Ell . The  remaining  Training  Session  11  flights  were 
flown  at  Carswell.  Specific  features  of  the  Carswell,  Barksdale,  and  Blytheville  runways  are  given  in  Table  4. 


Table  4.  Features  of  Carswell,  Barksdale,  and  Blytheville  Runways 


Bat* 

Runway 

WMth 

ILS 

VASI 

Barksdale 

14 

11,754' 

300' 



2.5° 

Blytheville 

18 

11,600' 

300' 

2.5° 

2.5° 

Carswell 

35 

12,000' 

300' 

2.5° 

2.75° 

17 

12,000' 

300' 

— 

2.75° 

Data  Collection 

Several  types  of  data  were  collected  throughout  the  study.  Examples  of  the  Student  Background 
Questionnaire,  Simulator  Usage  Log,  Student  Progress  Record,  and  the  Data  Card  are  given  in  Appendix  C. 

The  major  data  collection  tool  was  the  Data  Card.  It  was  used  to  score  performance  on  the  landing 
task  in  the  simulator,  as  well  as  in  the  aircraft.  The  card  is  a combination  of  a graphic  representation  of  the 
ground  track  of  the  landing  task,  flight  parameters  recorded  horn  flight  instruments,  and  letter  scores  on 
the  segments  of  the  landing  task  itself.  The  landing  task  was  divided  into  five  such  segments',  (a)  base  leg, 
(b)  turn  to  final,  (c)  final  approach  prior  to  descent,  (d)  final  approach  glide  path,  and  (e)  flare,  landing 
attitude,  and  touchdown.  Each  segment  and  its  scoring  are  described  in  Appendix  C. 

Two  other  descriptions  of  performance  in  the  aircraft  were  collected.  Following  each  landing,  the 
aircraft  commander  re-entered  the  traffic  pattern  and  proceeded  to  the  base  leg  start  point.  As  soon  as  he 
had  re-entered  the  pattern,  traffic  and  communications  permitting,  he  would  give  a segment-by-segment 
oral  evaluation  of  the  landing.  This  evaluation  was  recorded  on  an  in-cockpit  tape  recorder  and  later 
transcribed.  It  was  transmitted  over  the  aircraft  intercom  and  could  not  be  heard  by  the  subject. 

A video  tape  recording  of  the  landing,  from  a ground  camera  site,  was  used  to  support  this  evaluation. 
The  camera  location  was  approximately  50  yards  off  the  side  of  the  runway  in  the  touchdown  zone 
(approximately  1 ,000'  from  the  runway  threshold).  The  camera  followed  the  aircraft  from  the  base  leg 
starting  point  all  the  way  to  touchdown.  A narrative  description  of  the  landing  was  broadcast  by  the  JSO 
over  a discrete  UHF  radio  frequency  and  dubbed  onto  the  audio  channel  of  the  video  tape.  This  description 
could  not  be  heard  by  the  subject. 

These  narrated  video  recordings  were  extremely  valuable  in  reconstructing  specific  problem  areas  of 
various  subjects.  They  were  often  reviewed  by  the  aircraft  crew  following  the  flight.  Subjects  were  not 
invited  to  review  these  tapes. 
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Missing  U.  - A few  data  items  were  not  collected.  Due  to  technical  difficulties,  the  or»  board  cockpit 
tape  recorder  did  not  record  aircraft  commander  evaluations  on  El  and  E2.  The  video  taping  of  landings 
was  not  accomplished  on  El.  For  E2-E12,  occasionally  a video  equipment  malfunction  resulted  in  a 
landing  task  not  being  recorded.  All  the  data  cards,  however,  were  recovered  for  each  rater  and  every 
landing. 

Letter  Scores.  Four  major  categories  of  accomplishment  were  used  to  describe  subject  performance 
on  landing  segments:  E = Excellent;  G = Good;  F = Fair;  and  U = Unsatisfactory.  In  addition  to  these 
categories,  four  gradations  were  used:  Fair  Minus  (F— ),  Fair  Plus  (F+),  Good  Minus  (G— ),  and  Good  Plus 
(G+).  Word  pictures  for  these  eight  scores  are  given  in  Table  5.  These  descriptors  were  anchored  against 
pilot  skill  levels  which  were  defined  in  conjunction  with  the  raters.  The  Fair  category  was  defined  as  the 
average  skill  of  a student  copilot  who  was  approximately  halfway  through  CCTS.  The  Good  category  was 
defined  as  the  average  skill  of  a CCTS  copilot  graduate,  which  was  the  proficiency  objective  of  the 
simulator  training.  The  Excellent  category  was  reserved  for  performance  which  equalled  that  of  an 
accomplished  pilot  with  substantial  experience  in  the  KC-135. 


Table  5.  Word  Picture  Definitions  of  Letter  Scores 


Scar* 

Numerical 

Equivalent 

Word  Picture 

E 

7 

As  good  as  an  accomplished  pilot;  very  accurate  on  all  parameters;  no  error;  no  area  of 
criticism;  no  followup  instruction  required ; high  degree  of  skill  demonstrated. 

G+ 

6 

Slightly  better  than  G;  represents  the  skill  level  of  the  upper  10%  of  CCTS  copilot 
graduates. 

G 

5 

Proficient;  the  skill  level  of  the  average  CCTS  copilot  or  the  new  pilot  in  the  squadron; 
could  have  some  minor  variations  in  performance  or  rough  edges  but  on  the  average  is 
at  this  point. 

G- 

4 

Approaching  proficiency  but  not  yet  consistently  performing  at  the  G level; 
momentary  deviations  outside  desired  parameters;  characteristic  of  lower  10%  of  CCTS 
copilot  graduates. 

F+ 

3 

Slightly  better  than  fair;  improving  but  still  rough;  some  parameters  could  be  within  or 
approaching  the  G-/G  ranges,  but  overall  performance  is  still  too  variable. 

F 

2 

This  is  the  category  characteristic  of  the  CCTS  Student  copilot  performance,  the  aver- 
age student  copilot  halfway  through  CCTS;  many  rough  areas;  parameters  can  generally 
be  held  within  large  tolerances,  but  only  with  considerable  variability. 

F- 

1 

Barely  able;  initial  stage  of  skill  acquisition;  poor;  weak;  many  parameters  significantly 
out  of  tolerance. 

U 

0 

Unable  to  perform  task 
or 

Dangerously  outside  of  parameters;  unsafe 
or 

Seriously  deficient  in  areas  of  airmanship,  judgment,  decision  making,  or  performance 
under  stress. 

in.  RESULTS 

All  30  subjects  completed  the  simulator  training  and  evaluation  flight  phases  of  the  study.  The 
following  analyses  examine  performance  in  each  of  these  phases  separately. 
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Simulator  Training 

The  IP  progress  reports  show  that  training  did  occur  during  the  2 days  of  simulator  instruction.  For 
28  subjects  there  occurred  an  improvement  in  the  specific  skills  required  to  accomplish  the  landing  task  as 
well  as  an  improvement  in  the  ability  to  integrate  these  skills  into  an  overall  acceptable  piloting  behavior. 
For  the  remaining  two  there  was  an  improvement  in  certain  specific  skills,  but  the  integration  of  these  skills 
was  often  weak  or  inconsistent. 

Four  indices  of  training  performance  in  the  simulator  were  recorded  and  subsequently  analyzed. 
These  included:  (a)  the  number  of  subjects  reaching  proficiency,  (b)  total  training  time  in  the  simulator,  (c) 
total  time  flying  the  simulator  (performing  the  task),  and  (d)  the  total  number  of  landing  tasks  completed 
in  the  simulator.  These  data  are  summarized  in  Tables  6 and  7. 


Table  6.  Number  of  Subjects  Not  Reaching  Proficiency 


Training 

Session 

Experimental  Group 

Day 

Night 

TV 

Total 

i 

1 (5) 

1 (5) 

1 (5) 

3 (15) 

ii 

2 (5) 

3 (5) 

4 (5) 

9 (15) 

Total 

3 (10) 

4 (10) 

5 (10) 

12  (30) 

Not*.  — Parentheses  indicate  the  number  of  subjects  trained  within  the 
appropriate  cell,  column,  or  row. 


* 


Table  7.  Group  Means  for  Simulator 
Training  Performance  Data 


Tralnlrx  Session 
I li 


Measure 

Day 

Night 

TV 

Day 

Night 

TV 

Total  Training  Time 

6.11 

6.06 

6.56 

6.54 

6.93 

7.21 

Total  Hying  Time 

3.93 

3.53 

3.89 

4.05 

4.05 

4.28 

Total  Landing  Tasks 

57.20 

67.40 

77.80 

81.20 

75.60 

81.40 

In  spite  of  the  apparent  improvement  in  overall  skill  ability,  not  all  of  the  subjects  reached 
proficiency  as  defined  on  page  9.  As  shown  in  Table  6,  12  of  30  (40%)  did  not  complete  five  successive 
unaided  landings  within  the  specified  tolerances.  Chi-square  analyses  indicated  significantly  fewer  proficient 
subjects  in  the  second  class  (XJ  = 5.00,  df  = 1 , p < .05)  although  no  significant  differences  among  the  three 
groups  (XJ  = .83,  df  = 2,  p < .50).  An  examination  was  subsequently  made  of  the  simulator  progress 
records  (IP  comments)  and  in-simulator  protocols  from  the  experimenter  log  to  see  if  specific  problem  areas 
could  be  detected  which  were  associated  with  the  demonstrated  lack  of  proficiency.  A summary  of  these 
examinations  for  the  Day,  Night  and  TV  nonproficiency  subjects  is  provided  in  Appendix  D. 

The  data  for  the  remaining  measures  of  training  performance  are  summarized  in  Table  7.  Each 
measure  was  analyzed  by  a completely  randomized  two-factor  analysis  of  variance  (visual  system  X training 
session).  An  analysis  of  the  total  training  time  in  the  simulator  revealed  a significant  main  effect  for  the 
training  session  factor  (F(l,24)=  4.66,  p < .05).  The  visual  system  main  effect  (F(2,24)  = 1.20)  as  well  as 
the  interaction  (F(2,24)  = .18)  were  nonsignificant.  An  inspection  of  Table  7 reveals  that  the  total  time  in 
the  simulator  was  greater  for  the  second  class.  For  those  individuals  not  reaching  proficiency,  time  in  the 
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simulator  was  maximum ; that  is,  7.2S  hours.  Since  the  second  class  had  a higher  percentage  of  students  not 
reaching  proficiency,  the  increase  in  total  training  time  is  easily  explained.  An  analysis  of  the  total  time 
spent  performing  the  task  revealed  a similar  trend  although  the  differences  were  not  statistically  significant. 
Main  effects  for  visual  system  (F(2,24)  = .80)  and  training  session  (F(2,24)  = 3.05)  as  well  as  their 
interaction  ( F(2 ,24)  = .35)  produced  no  statistically  reliable  differences. 

An  analysis  of  the  total  number  of  landings  performed  yielded  a significant  main  effect  for  the 
training  session  factor  (F(l,24)  = 10.72,  p < .01),  but  non-significant  results  for  the  visual  system  factor 
(F(2,24)  = 3 .00)  and  their  interaction  (F(2,24)  = 2 .87).  Despite  the  lack  of  statistical  significance,  there  is  a 
trend  toward  differences.  Table  7 shows  a decreased  number  of  repetitions  for  the  first  class  of  the  Day 
group  despite  the  fact  that  time  in  the  simulator  was  approximately  the  same  as  that  for  the  Night  group. 
This  was  due  to  an  error  made  in  determining  the  reset  coordinates  for  starting  the  landing  task  in  the  Day 
simulator.  The  TV/Model  board  was  the  limiting  factor  in  the  distance  from  the  runway  from  which  a 
landing  task  could  be  started.  The  TV  probe  could  move  only  so  far  into  the  comer  of  the  board  before  it 
moved  off  the  board  entirely.  Therefore,  the  Night  and  Day  reset  points  were  made  to  match  the  farthest 
extended  TV  reset  point  to  achieve  the  longest  possible  standardized  pattern.  An  error  had  been  made  in 
calculating  the  coordinates  for  the  Day  system  and  the  Day  landing  task  was  slightly  farther  out  with  a 
longer  base  leg  than  the  Night  and  TV  landing  tasks.  This  was  not  detected  until  after  the  completion  of 
Training  Session  1.  On  the  average,  Dl  -D5  flew  approximately  a 4.0-minute  landing  task  while  N1-N5  and 
TV1-TV5  flew  approximately  a 3.0-minute  landing  task.  The  Day  coordinates  were  corrected  for  Training 
Session  II  and  all  subjects  flew  approximately  a 3.0-minute  pattern.  While  D1-D5  flew  fewer  discrete 
landing  tasks.  Table  7 shows  that  their  total  time  flying  the  simulator  (performing  landing  tasks)  compared 
with  their  total  time  in  the  simulator  was  greater  (64%)  than  the  Session  I Night  group  (58%)  or  TV  group 
(59%). 

An  examination  of  the  total  number  of  landing  tasks  accomplished  in  the  typical  1.5-hour  training 
period  dispels  any  concern  that  the  TV  system  would  not  allows  as  many  landing  tasks  because  of  the  extra 
time  for  it  to  reset  itself  due  to  the  slow  moving  gantry.  All  three  systems  averaged  about  19  landing 
tasks/1 .5-hour  period,  depending  on  instructor  technique,  use  of  problem  freeze  and  length  of  between-task 
critiques. 

The  insertion  of  turbulence  into  the  landing  task  during  Period  2 had  a detrimental  effect  on  subject 
performance  for  all  groups.  The  erratic  motion  added  a new  dimension  to  the  control  problem  that  required 
additional  time  to  master.  The  introduction  of  crosswinds  had  a similar  effect.  The  IPs  felt  that  the  addition 
of  the  turbulence  and  crosswind  motion  component  into  the  overall  cockpit  motion  formula  added  to  the 
value  of  the  training. 

Evaluation  Flights 

Twelve  KC-135  aircraft  flights  were  flown  to  evaluate  transfer  of  visual  system  training  to  aircraft 
performance.  During  these  flights  209  approaches  and  landings  were  attempted  and  197  were  completed. 
Twelve  ended  prematurely  with  a go-around  resulting  from  traffic  conflicts,  excessive  crosswinds,  or  other 
reasons.  Twenty -seven  subjects  made  six  or  more  total  landings  over  both  pairs  of  evaluation  flights.  Three 
subjects  made  six  approaches  over  both  days,  but  only  five  resulted  in  touchdowns. 

Subject  performance  on  each  landing  was  evaluated  in  two  steps.  First,  data  cards  were  used  on  board 
by  the  JSO  and  IPs  to  score  each  of  the  five  segments  of  the  landing.  These  segment  scores  were 
transformed  and  combined  into  composite  landing  scores,  providing  a comparison  of  overall  landing 
performance  as  well  as  a segment-by -segment  analysis.  Second,  the  narrative  evaluations,  supported  by  the 
video  tape  recordings  of  the  landings,  were  used  to  determine  the  types  of  errors  the  subjects  made,  thereby 
suggesting  reasons  why  a particular  score  was  given. 

Landing  Scores.  Each  landing  was  graded  (using  the  data  card)  by  JSO  and  the  IPs  on  board  the 
aircraft.  Raters  individually  judged  each  segment  of  the  landing  and  recorded  a segment  score  on  the  card. 
Each  score  was  converted  from  the  rater’s  letter  grade  score  to  its  numerical  equivalent.  The  decision  to 
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comu;  te  JSO  and  IP  segment  scores  was  made  after  rank -orderings  of  daily  subject  performance  indicated  a 
high  legree  of  agreement  among  these  raters.  Following  each  evaluation  flight,  each  rater  independently 
rank-ordered  the  five  subjects  according  to  the  skill  demonstrated.  To  assess  the  degree  of  inter-rater 
agreement,  the  Kendall  coefficient  of  concordance,  W,  was  computed  for  each  evaluation  flight.  The 
computed  W’s  ranged  from  .76  to  1.00  with  an  average  of  .87.  All  of  the  coefficients  were  statistically 
significant,  indicating  substantial  agreement  among  the  raters.  Consequently,  the  composite  segment  score 
for  each  landing  was  computed  to  be  the  mean  of  the  individual  segment  scores  across  all  raters.  A daily 
score  for  each  segment  was  then  computed  by  taking  the  mean  of  the  composite  segment  scores  for  the  first 
three  landings.  These  daily  individual  segment  scores  were  used  as  the  raw  data  for  all  subsequent  analyses. 
They  are  summarized  in  Table  8. 


Table  8.  Group  Means  for  Evaluation  Flight  Data 


Stgmint 

Evaluation  Flljht 

1 

II 

Day 

Night 

TV 

Day 

Night 

TV 

Base  Leg 

4.89 

4.86 

4.92 

5.28 

5.38 

5.08 

Turn  to  Final 

4.13 

3.60 

3.69 

4.43 

4.56 

3.62 

Prior  to  Descent 

4.28 

4.66 

4.49 

4.98 

4.85 

3.88 

Glide  Path 

3.62 

4.11 

2.81 

433 

4.44 

3.05 

Landing 

2.70 

3.30 

2.50 

3.65 

4.13 

2.10 

The  data  were  analyzed  by  a three  factor  analysis  of  variance  having  one  between-groups  factor 
(visual  system)  and  two  within-groups  factors  (evaluation  flight  and  maneuver  segment).  The  results  of  the 
analysis  are  presented  in  Table  9.  As  indicated,  all  main  effects  were  statistically  significant,  as  well  as  two 
of  the  first-order  interactions. 


Table  9.  Analysis  of  Variance  Summary  Table 


Source 

ss 

df 

Ms 

F 

Between  Subjects 

161.5519 

29 

A 

33.4817 

2 

16.7409 

3.5294** 

Subj.  w/Grp. 

128.0702 

27 

4.7433 

Within  Subjects 

375.9340 

270 

B 

9.0134 

1 

9.0134 

8.9963*** 

AB 

8.7603 

2 

4.3802 

4.3719** 

B X SwGrp. 

27.0523 

27 

1.0019 

C 

140.4278 

4 

34.0609 

36.2602*** 

AC 

14.6726 

8 

1.8341 

1.8968* 

C X SwGrp. 

104.4278 

108 

.9669 

BC 

1.2902 

4 

.3226 

.5310 

ABC 

4.8601 

8 

6V'S 

1.0000 

BC  X SwGrp. 

65.6137 

108 

.6075 

Total 

537.4859 

299 

Not*.  — Factor  A - Visual  System. 

Factor  B - Evaluation  Flight. 
Factor  C - Maneuver  Segment. 

*P<.I0. 

••P<.05. 

•*«P<.01. 
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The  data  revealed  an  overall  difference  in  performance  across  the  three  groups  trained  with  the 
different  visual  systems.  A Tukey’s  Honestly  Significant  Difference  (HSD)  test  for  pairwise  comparisons 
indicated  a significant  difference  for  the  Night  versus  TV  comparison  (p  < .05)  and  also  a difference  for  the 
Day  versus  TV  comparison  (p  < .10).  No  differences  were  found  for  the  Day  versus  Night  comparison. 
Using  the  Sheffe’s  S method,  the  data  from  the  Day  and  Night  groups  were  combined  and  found  to  be 
superior  to  the  performance  of  the  TV  group  (p  < .05). 

The  other  significant  main  effects  were  originally  expected  and  provide  support  to  the  validity  of  the 
study  results.  First,  the  data  revealed  a significant  improvement  in  performance  from  the  first  to  the  second 
evaluation  flight.  Second,  the  data  indicated  significant  differences  in  performance  across  the  five  segments 
of  the  landing  tasks.  The  first  three  segments  require  the  pilot  to  maintain  straight-and -level  flight,  execute 
a level  turn,  and  again  maintain  level  flight  until  the  beginning  of  the  descent.  The  fourth  and  fifth  segments 
are  considered  to  be  much  more  difficult  since  they  require  the  pilot  to  make  his  final  descent,  flare,  and 
touchdown.  The  data  from  Table  8 are  consistent  with  this  fact  and  clearly  demonstrate  a performance 
decrement  in  the  final  two  segments  of  the  landing  pattern.  Likewise,  a level  turn  is  considered  to  be  more 
difficult  than  maintaining  straight-and-level  flight,  a supposition  which  is  also  demonstrated  by  the  data. 
The  correspondence  of  the  data  to  these  expectations  provide  evidence  of  the  internal  validity  of  the  study 
effort. 

To  evaluate  the  significant  visual  system  by  evaluation  flight  interaction,  tests  for  simple  main  effects 
were  computed.  The  results  indicated  a significant  effect  across  the  three  visual  systems  on  the  second 
evaluation  flight  (F(2,27)  = 6.56,  p<  .01),  but  not  the  first  (F(2,27)  = .79).  Likewise,  a significant 
improvement  in  performance  from  the  first  to  the  second  evaluation  flight  was  obtained  for  the  Day  group 
(F(l,27)  = 9.28,  p < .01)  and  Night  group  (F(l,27)  = 7.99,  p < .01),  but  not  for  the  TV  group  (F(l,27)  = 
.46).  These  data  are  graphically  presented  in  Figure  1. 
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Simple  main  effect  tests  were  also  computed  for  the  visual  system  by  maneuver  segment  interaction. 
No  differences  in  performance  across  the  three  groups  were  obtained  for  Base  Leg  (F(2,27)  = .04),  Turn  to 
Final  (F(2,27)  = 1.18),  and  Final  Approach  Prior  to  Descent  (F(2,27)=  1.04)  segments  of  the  landing. 
Differences  did  emerge  for  the  Final  Approach  Glide  Path  (F(2,27)  = 5.79,  p < .01)  and  Rare,  Landing 
Attitude,  and  Touchdown  (F(2,27)  = 5.92,  p<  .01)  segments.  Significant  differences  across  the  five 
segments  were  obtained  for  all  three  groups  - Day  (F(4,108)  = 10.36,  p < .01),  Night  (F(4,108)  = 6.15, 
p < .01),  and  TV  (F(4,108)  = 22.22,  p < .01).  These  data  are  graphically  summarized  in  Figure  2. 


1 2 3 4 5 

Maneuver  Segments 


Figure  2.  Group  means  across  maneuver  segments. 


The  analysis  of  the  performance  data  collected  on  the  two  aircraft  evaluation  flights  combined  the 
subjects  from  the  two  classes.  In  the  analysis  of  the  simulator  training  data,  significant  differences  emerged 
between  these  classes.  A significantly  greater  proportion  of  students  in  the  first  training  session  reached  the 
proficiency  criteria.  The  question  arose  whether  these  differences  were  reflected  in  the  transfer  data  and 
more  importantly,  whether  there  existed  a class  by  visual  system  interaction.  In  the  event  of  a significant 
interaction,  the  validity  of  combining  the  data  across  the  two  classes  would  be  questionable. 

For  each  subject  an  overall  score  was  computed  by  taking  the  mean  of  the  individual  segment  scores 
across  the  two  evaluation  flights.  These  data  are  summarized  in  Figure  3.  A completely  randomized 
two-factor  analysis  of  variance  (visual  system  X training  session)  was  computed.  The  results  indicated 
significant  main  effects  for  the  visual  system  factor  (F(2,24)  = 4.07,  p < .05)  and  the  training  session  factor 
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Figure  3.  Group  means  across  training  sessions. 

(F(l,24)  = 5.06,  p < .05)  but  not  for  their  interactions  (F(2,24)  = 1 .04).  As  shown  in  Figure  3,  subjects  in 
the  first  class  performed  significantly  better.  Calculations  of  omega-squared  indicated  that  the  training 
session  further  accounted  for  10.08%  of  the  total  variance  of  the  dependent  measure  while  the  visual 
system  factor  accounted  for  15.25%. 

Frequencies  and  Types  of  Errors.  The  aircraft  commander  narrative  evaluation  combine J with  the 
video  tape  records  were  used  to  perform  an  error  analysis  on  the  types  and  frequencies  of  errors  on  each  of 
the  segments.  This  analysis  identifies  the  specific  causes  for  the  scores  reported  in  the  previous  section.  Of 
special  interest  was  whether  there  were  any  trends  among  groups  that  could  be  indicative  of  a visual  system 
weakness. 

1.  Base  Leg.  There  were  very  few  problems  with  maintaining  straight-and-level  flight  on  the  base  leg. 
There  were  no  apparent  differences  between  groups. 

2.  Turn  to  Final.  Table  10  shows  the  proportion  of  turns  where  altitude  control  problems  and 
roll -out  problems  occurred. 

Altitude  was  not  maintained  on  approximately  29%  of  the  turns  made  by  all  subjects.  For  the 
majority  of  these  turns  the  loss  or  gain  was  in  the  100’  to  150’  range.  The  Day  group  experienced  both 
losses  and  gains,  while  the  Night  and  TV  groups  experienced  only  losses  in  altitude.  Alignment  with  the 
runway  centerline  upon  rolling  out  was  a problem  on  40%  of  all  turns.  Undershooting  occurred  about  twice 
as  often  as  overshooting  for  all  three  groups. 
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Table  10.  Error  Analysis  for  Turn  to  Final 


Oay  Nl«ht 

Proportion  of  Group  Turns  with  Loss  or  Gain  in  Altitude 

Altitude  Loss  .17  .35 

Altitude  Gain  .10 


.25 


Proportion  of  Group  Turns  with  Undershooting  or  Overshooting 


Undershot  .32  .23  .27 

Overshot  .17  .12  .10 


3.  Final  Approach  Prior  to  Descent.  No  significant  problems  occurred  during  this  portion  of  the 
landing. 

4.  Final  Approach  Glide  Path.  The  decision  to  begin  the  descent  was  more  often  late  (22%)  than 
early  (7%)  for  the  subjects  overall,  although  71%  of  the  descents  were  began  at  the  proper  point.  Of  the 
Day  group  descents,  22%  were  late,  but  none  early.  Of  the  Night  group  descents,  18%  were  late  and  15% 
were  early.  Of  the  TV  group  descents,  27%  were  late  and  5%  were  early. 

Deviations  from  the  glide  path  were  tallied  for  each  third  of  the  final  approach.  (Figure  4 shows  the 
proportion  of  approaches  by  students  within  each  group  that  were  considered  “on  glide  path”  for  each 
third  of  the  final  approach.)  The  Night  group  was  consistently  on  glide  path  50  to  55%  of  the  time.  The 
Day  and  TV  groups  had  fewer  “on  glide  path”  approaches. 

Figure  4 also  shows  the  proportion  of  approaches  that  were  either  high  or  low.  The  vertical  axis  of 
each  graph  indicates  the  proportion  of  glide  paths  that  deviated  the  specified  amount  from  the  glide  path. 
The  horizontal  axis  of  each  graph  represents  the  severity  of  the  deviation,  with  H or  L representing  a 
deviation  less  than  one  dot  width  on  the  ILS,  *H  or  *L  indicating  a deviation  of  one  to  two  dot  widths,  and 
**H  or  **L  indicating  more  than  two  dot  widths  of  deviation. 


Figure  4 shows  that  the  Day  group  tended  to  be  high  for  the  first  two  thirds  of  the  glide  path  and 
then  low  on  the  final  third.  The  Night  group  was  higher  on  the  first  two  thirds  but  fairly  balanced  on  the 
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final  third,  with  fewer  extreme  deviations  (**H  and  **L).  The  TV  group  tended  to  be  higher  on  the  first 
third,  more  balanced  on  the  second  third,  and  considerably  low  on  the  final  third,  with  substantially  more 
extreme  deviations  than  the  other  two  groups.  Across  the  entire  length  of  the  glide  path,  4%  of  the  Day 
glide  path/thirds  had  extreme  deviations,  less  than  1%  of  the  Night  thirds  had  extreme  deviations,  and  14% 
of  the  TV  thirds  had  extreme  deviations. 

5.  FI are.  Landing  Attitude,  and  Touchdown . The  flare  was  a problem  on  35%  of  the  Day,  35%  of 
the  Night,  and  58%  of  the  TV  landings.  This  included  flaring  high,  low,  or  not  at  all.  Each  group  had  a 
moderate  portion  of  skips,  bounces,  and  firm  touchdowns  (Day  25%,  Night  23%,  TV  32%). 

For  2091  of  the  Day  landings,  the  aircraft  commander  had  to  make  a control  input,  ranging  from  a 
blocking  action  on  the  yoke  to  prevent  the  aircraft  nose  from  raising  too  high  to  a deliberate  aileron  input 
to  neutralize  wind  rock.  Control  inputs  were  made  on  13%  of  the  Night  landings  and  27%  of  the  TV 
landings.  The  aircraft  commander  had  to  take  the  aircraft  from  the  subject  due  to  determining 
circumstances  (approaching  the  outer  limits  of  tolerances)  on  10%.  of  the  Day  landing,  8%  of  the  Night 
landings,  and  12%  of  the  TV  landings. 

CCTS  Training  Performance 

The  progress  of  subjects  participating  in  the  study  was  monitored  throughout  CCTS.  After  arriving  at 
CCTS,  the  subjects  began  a 5-week  academic  block  of  instruction  with  no  opportunity  to  fly  the  aircraft. 
When  they  did  go  to  the  flight  line,  they  had  to  transition  from  the  left  seat  (as  trained  in  this  study)  to  the 
right  seat.  Each  pilot  flew  as  part  of  a crew  on  nine  training  sorties,  one  check  ride,  and  one  solo  sortie. 
During  all  of  this  flying  the  student  made  about  20  landing.  On  the  check  ride  given  at  the  end  of  training, 
students  were  evaluated  as  being  Highly  Qualified,  Qualified,  Qualified  with  Additional  Training,  or 
Unqualified.  All  subjects  participating  in  the  study  were  rated  Qualified  or  Highly  Qualified.  The 
breakdown  according  to  group  is  presented  in  Table  1 1. 


Table  1 1.  Final  CCTS  Check  Ride  Evaluations 


Evaluation 

Day 

Night 

TV 

Highly  Qualified 

9 

5 

4 

Qualified 

1 

5 

6 

Thus  far,  the  results  have  focused  on  a comparison  of  performance  among  the  three  groups  trained 
with  different  visual  systems.  Since  the  study  did  not  have  a control  group  without  simulator  training,  it 
seemed  necessary  to  compare  the  present  subjects'  performance  with  CCTS  student  copilots  from  other 
classes.  One  source  of  comparison  was  the  judgment  of  the  aircraft  commander  who  conducted  the 
evaluation  flights.  Comparing  subject  performance  on  the  evaluation  flights  with  the  skill  levels  of  the 
typical  student,  it  was  his  judgment  that  the  majority  of  the  experimental  subjects  performed  at  a skill  level 
comparable  to  a CCTS  student  copilot  well  along  in  CCTS  training.  Further,  several  of  the  subjects 
consistently  executed  the  landing  task  as  well  as  the  average  CCTS  copilot  graduate,  and,  on  a number  of 
occasions,  individual  landings  were  executed  on  a skill  level  equaling  an  experienced  KC-135  command 
pilot.  This  observation  was  supported  by  the  other  IPs  on  board  the  aircraft  and  on  the  ground. 

An  additional  source  of  comparison  comes  from  the  final  check  ride  evaluations  of  students  from 
other  classes.  Data  were  compiled  from  the  seven  classes  preceding  the  two  experimental  classes  and  the 
first  class  thereafter.  The  two  categories.  Qualified  with  Training  and  Unqualified,  were  combined  due  to 
the  small  number  of  students  receiving  these  evaluations.  The  resulting  data  are  presented  in  Table  12.  As 
shown,  60%  of  the  pilots  in  the  two  experimental  classes  received  a Highly  Qualified  evaluation,  whereas 
only  30%  of  students  in  the  other  eight  classes  received  this  rating.  A chi-square  test  indicated  these 
differences  to  be  statistically  significant  (XJ  = 10.12,  df  = 2,  p < .01). 
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Fable  12.  Check  Ride  Comparison  with  Other  Classes 


Evaluation 

Study  Clauat 

Othar  Ctaua* 

Highly  Qualified 

18 

(607!) 

38  (30.4%) 

Qualified 

12 

(40%) 

78  (62.4%) 

Not  Qualified 

0 

9 (7.2%) 

tv.  DISCUSSION 

Four  topics  are  addressed  in  this  section:  (a)  the  transfer  effectiveness  of  the  simulation  training,  (b) 
the  relative  efficiency  of  the  simulation  training  across  the  three  visual  systems,  (c)  the  relative  effectiveness 
of  the  simulation  training  across  the  three  visual  systems,  and  (d)  the  effective  utilization  of  visual 
simulation  in  operational  training. 

Transfer  Effectiveness  of  Simulation  Training 

The  results  clearly  demonstrate  the  transfer  effectiveness  of  the  training  provided  in  the  study.  This 
conclusion  is  supported  by  the  subjective  opinion  of  the  IPs  as  well  as  performance  on  the  final  CCTS 
evaluation  check  ride.  In  the  eight  other  classes  used  for  comparison,  30  percent  received  a final  evaluation 
of  Highly  Qualified.  For  the  two  classes  participating  in  the  present  study,  60  percent  were  rated  at  this 
level.  Despite  these  figures,  care  must  be  exercised  in  their  interpretation.  Since  these  evaluations  represent 
terminal  performance  at  the  end  of  training,  there  is  no  information  reflecting  time  to  proficiency. 
Consequently,  the  usual  measures  of  transfer  effectiveness  are  not  available. 

During  CCTS,  each  student  accomplished  approximately  20  landings  in  the  aircraft.  During  the 
experimental  training,  each  student  accomplished  approximately  73  landings  in  the  simulator  and  six 
landings  in  the  aircraft.  Since  the  number  of  landings  accomplished  on  the  two  evaluation  flights  represent  a 
sizeable  proportion  (30  percent)  of  the  total  accomplished  in  CCTS,  a precise  estimate  of  the  effects  solely 
attributable  to  the  simulation  training  is  not  available.  Despite  this  confounding  and  the  lack  of  time  to 
proficiency  information,  the  data  clearly  indicate  that  transfer  of  training  did  occur.  Subjects  in  the  two 
experimental  classes  who  received  the  simulator  pretraining  did  perform  significantly  better  than  students 
from  other  classes.  It  is  only  the  magnitude  of  these  effects  that  remains  undetermined. 

Relative  Efficiency  of  Visual  Simulation  Training 

The  training  efficiency  of  a training  device  can  be  measured  by  the  time  needed  to  train  a specific 
task.  The  total  training  time  for  each  group  (Table  7)  is  one  index  of  the  efficiency  of  each  visual  system. 
These  data  do  not  show  any  statistically  reliable  differences  between  visual  systems.  The  difference  between 
the  shortest  and  longest  mean  times  to  train  the  landing  task  was  only  33  minutes.  However,  this  table  uses 
7.25  hours  as  the  total  time  for  subjects  who  did  not  achieve  proficiency.  If  each  subject  had  been  allowed 
to  take  as  much  time  as  needed  to  satisfy  the  proficiency  criterion,  more  definitive  trends  might  have 
appeared.  Using  the  “Estimated  Time  to  Proficiency”  from  Appendix  D with  3 hours  estimated  for  N6. 
these  times  can  be  projected,  if  only  roughly.  The  resulting  group  means  are  still  nonconclusive : 6.63  (Day), 
7.10  (Night),  and  7.48  (TV).  The  difference  between  the  Day  and  TV  is  51  minutes.  This  could  be 
indicative  of  a trend,  but  remains  a very  small  amount  of  time. 

Even  if  the  results  had  indicated  a more  dramatic  difference  in  training  efficiency,  it  would  be 
difficult  to  speculate  on  the  impact  of  such  a finding.  Since  the  simulator  is  a training  device  and  an  integral 
part  of  a training  system,  it  is  the  efficiency  of  the  overall  system  which  is  of  real  concern.  The  efficiency  of 
each  component  of  the  system  is  dependent  upon  other  components  in  the  system  and,  therefore,  the 


22 


* 


efficiency  of  the  simulator  will  be  dependent  on  how  it  is  used,  where  it  is  sequenced  into  die  training 
syllabus  along  with  other  training  devices  (including  the  aircraft,  which  is  also  a training  device),  how  often 
it  is  used  and  for  how  long,  how  it  is  viewed  oy  the  user  (IP,  as  well  as  the  student),  and  many  other  factors. 
Had  one  of  the  visual  systems  shown  a marked  lack  of  efficiency  under  the  conditions  of  this  study,  it 
could  be  argued  that  a simple  rearrangement  in  the  simulator  schedule  or  a revised  sequencing  of  simulator 
and  aircraft  sorties  would  adequately  remove  the  deficiency.  Therefore,  the  final  statement  of  trainer 
efficiency  must  be  made  within  the  context  of  the  total  training  system  after  each  component  is  studied. 

Relative  Effectiveness  of  Visual  Simulation  Training 

While  efficiency  concerns  the  time  to  train  a task  in  the  simulator,  effectiveness  reflects  the  impact  of 
that  training  upon  subsequent  performance  in  the  aircraft.  The  data,  presented  in  Table  8,  with  the 
supporting  statistical  analyses,  indicated  that  the  subjects  in  the  Day  and  Ni^it  groups  performed 
significantly  better  than  those  in  the  TV  group.  These  performance  differences  were  reflected  primarily  in 
the  last  two  segments  of  the  landing  task,  the  Final  Approach  Glide  Path  and  Flare,  Landing  Attitude, and 
Touchdown.  Furthermore,  the  differences  among  groups  were  statistically  significant  only  on  the  second 
evaluation  flight.  The  Day  and  Night  groups’  performance  improved  on  the  second  evaluation  flight  whereas 
the  performance  of  the  TV  group  remained  approximately  the  same.  The  reason  behind  the  failure  of  the 
TV  group  to  improve  their  performance  on  the  second  flight  is  unknown.  It  may  be  speculated  that  the 
training  provided  by  the  TV  system  was  not  sufficient  to  enable  the  student  to  profit  from  his  initial  three 
landings  in  the  aircraft.  In  other  words,  the  entry  skill  level  of  these  students  may  have  been  in  the  lower 
end  of  the  learning  curve  where  no  improvement  in  performance  is  often  observed. 

The  error  analyses  of  segment  performance  indicated  that  the  base  leg  did  not  present  any  significant 
problems,  probably  since  the  aircraft  commander  had  already  established  the  aircraft  in  a steady  state  and 
the  subject  simply  had  to  maintain  it.  Similarly,  while  subjects  had  altitude  and  centerline  problems  in  the 
turn,  these  problems  were  not  major.  For  the  majority  of  these  errors,  the  subjects  realized  they  were  out 
of  tolerance  and  proceeded  to  make  the  proper  corrections. 

The  final  approach  glide  path  was  consistently  the  most  difficult  area  of  performance.  The  subject 
had  to  rely  on  the  runway  picture  and  an  approximate  rate  of  descent  reference  from  the  vertical  velocity 
indicator  as  his  primary  cues.  As  shown  in  Figure  4,  the  Night  group  was  “on  glide  path”  50  to  55  percent 
of  the  time,  while  the  Day  group  was  “on  glide  path”  37  to  50  percent  and  the  TV  group  was  “on  glide 
path”  35  to  42  percent.  The  Day  and  TV  groups  were  on  glide  path  more  in  the  middle  third  of  the 
approach  than  the  first  or  final  third,  while  the  Night  group  was  more  accurate  on  the  final  third  of  the 
glide  path. 

All  three  groups  tended  to  be  high  rather  than  low  on  the  first  third  of  the  approach  (Figure  4).  For 
the  second  third,  the  Day  group  remained  almost  exclusively  high  while  the  Night  and  TV  groups  were 
more  balanced.  For  the  final  third  the  Night  groups  remained  approximately  balanced,  but  the  Day  group 
went  low  and  the  TV  group  was  even  lower.  What  is  most  striking  in  Figure  4 is  the  proportion  of  critical 
deviations  in  excess  of  two  dot  widths  (**H  and  **L)  especially  in  the  final  third  of  the  approach.  The 
Night  group  had  very  few  critical  deviations,  foDowed  closely  by  the  Day  group.  The  TV  group  had  a 
substantial  number  of  critical  deviations,  especially  in  **L  (in  excess  of  two  dots  low)  during  the  final 
one-third  of  the  approach. 

In  the  final  segment,  many  flares  were  started  too  high,  and  a number  were  started  too  late,  with  the 
aircraft  impacting  the  runway  without  establishing  a landing  attitude.  The  higher  percentage  of  poor  flares 
for  the  TV  group  (58  percent)  is  mostly  from  the  higher  percentage  of  high  and  low  final  glide  paths.  This 
carried  over  to  skips,  bounces,  and  firm  touchdowns. 

The  aircraft  commander  had  to  take  the  aircraft  from  the  subject’s  control  on  aoout  the  same  number 
of  landings  for  all  groups.  This  action  was  precipitated  by  the  aircraft  approaching  the  outer  limits  of 
tolerances,  for  example  excessive  wing  rock  close  to  the  ground,  too  low  on  final  approach,  or  a firm 
touchdown  and  bounce  into  an  airborne  condition.  Counting  these  occasions  with  other  aircraft 
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commander  inputs  (preventing  the  yoke  from  coming  b?-'-  too  far,  an  aileron  input  to  level  the  wings,  etc.), 
the  Night  group  had  fewer  total  aircraft  commander  r t;  ■ ’3  percent)  than  the  Day  group  (20  percent)  or 
TV  group  (27  percent). 

Effective  Utilization  of  Visual  Simulation 

There  is  evidence  to  suggest  that  how  a training  device  is  used  often  accounts  for  more  training 
output  (efficiency,  as  well  as  effectiveness)  than  the  hardware  characteristics  of  the  device.  Sophisticated 
hardware  refinements  of  a device  might  result  in  only  token  increases  in  training  effectiveness,  whereas 
simple  innovations  in  the  use  of  the  device  might  substantially  increase  its  effectiveness.  Some  of  the 
observations  made  by  the  research  and  ins»ructor  staff  concerning  such  applications  are  presented  below. 

For  the  task  trained  in  this  study,  the  1.5-hcur  training  period  seemed  to  be  too  long.  A 1.0-hour 
period  probably  would  have  been  more  beneficial  to  the  training  objective.  Reports  of  mental  fatigue  were 
received  from  some  subjects.  Eye  fatigue  was  also  reported,  partly  due  to  the  type  of  optics  used  to  display 
the  visual  model.  Occasionally  eye  discomfort  was  caused  by  the  instant  change  in  brightness  when  the  Day 
simulator  reset  itself.  Visual  system  manufacturers  should  consider  a less  drastic  light  intensity  change  when 
0 ‘signing  the  reset  function  in  their  devices. 

The  sequencing  of  training  devices  in  a curriculum  should  be  based  on  a scheme  matching  the  device 
or  media  to  the  learner’s  state  of  preparedness  to  leam.  The  progression  from  a textbook  to  the  learning 
center,  a cockpit  procedures  trainer,  a part-task  trainer,  a simulator,  and  finally  the  airplane,  should  have 
some  rationale  and  be  empirically  tested  for  a given  task  and  student  population.  Eddowes  ( 1974)  describes 
such  a training  strategy. 

In  the  present  study  a cognitive  pretraining  technique  of  the  type  being  investigated  by  Edwards 
(1978)  might  have  reduced  the  overall  simulator  training  time.  In  his  current  study,  Edwards  shows  pictures 
of  various  landing  attitudes  and  aircraft  positions  taken  from  the  cockpit  prior  to  the  student’s  aircraft 
flight.  The  objective  is  for  the  student  to  learn  the  visual  discrimination  skills  he  needs  to  land  the  aircraft 
before  he  goes  to  fly.  A similar  set  of  materials  might  have  been  useful  to  demonstrate  the  correct  runway 
picture  to  subjects  before  they  entered  the  simulator,  thus  saving  expensive  simulator  time  and  optimizing 
instructor  and  student  efforts.  The  sequencing  of  the  aircraft  should  also  be  carefully  studied  in  the  overall 
curriculum  design.  The  aircraft,  like  the  other  components  of  the  training  system,  should  be  introduced  to 
the  student  at  that  point  where  it  optimizes  its  training  benefit. 

One  instructional  technique  which  was  used  successfully  in  the  simulator  was  problem  freeze. 
Problem  freeze  was  used  frequently  to  stop  the  simulator  in  mid-approach  and  point  out  aspects  of  the 
landing  picture  to  the  student.  The  instructor  could  select  various  parameters  for  freezing  and  leave  others 
alone.  For  example,  on  short  final,  position  freeze,  and  altitude  freeze  were  used.  The  instructor  could  then 
pivot  the  simulator  as  if  it  were  on  a ball  and  show  the  student  nose  low  and  high  attitudes,  improper 
alignment  and,  by  moving  the  airplane  vertically  with  altitude  freeze  off  momentarily,  he  could  show  high 
and  low  glide  path  conditions.  Likewise,  the  insertion  of  crosswinds  and  turbulence  was  also  considered 
beneficial  to  the  simulator  training. 

Although  little  data  are  available,  it  is  widely  believed  that  simulators  can  be  more  effectively  utilized 
if  students  are  trained  to  proficiency  before  they  are  advanced  to  the  aircraft.  The  data  collected  in  the 
present  study  permitted  a comparison  of  performance  in  the  aircraft  between  those  who  did  and  did  not 
reach  proficiency  in  the  simulator.  Eighteen  subjects  were  considered  proficient  while  12  were  not.  For 
each  subject,  the  mean  score  across  the  two  evaluation  flights  was  used  as  an  estimate  of  performance  in  the 
aircraft.  A t-test  indicated  performance  of  the  “proficient”  group  to  be  somewhat  better  (t  = 1 .82,  df  = 28, 
p<  .10).  Although  these  results  are  not  striking,  they  do  provide  evidence  in  support  of  the  proficiency 
advancement  concept. 
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V.  CONCLUSIONS 


The  three  visual  systems,  evaluated  in  this  study,  were  successful  in  increasing  the  skill  level  of 
student  copilots  during  a 2-day  training  period.  When  compared  with  other  student  copilots,  the  simulator 
trained  students  performed  at  a higher  skill  level  at  CCTS. 

A detailed  examination  of  the  training  efficiency  of  each  visual  system  did  not  uncover  any  reliable 
differences  between  systems.  However,  the  Day  and  Night  systems  trained  more  effectively  than  the  TV 
system,  as  evidenced  by  reliably  higher  evaluation  flight  scores  for  Day  and  Night  subjects.  Neither  the  Day 
nor  Night  system  was  more  effective  than  the  other.  The  TV  system  trained  less  effectively  in  the  final 
approach  glide  path  segment  of  the  landing  task,  with  proportionately  more  extreme  deviations  from  the 
glide  path,  especially  with  very  low  glide  slopes  in  the  final  one-third  of  the  approach.  The  Day  group  also 
tended  to  be  low  in  the  final  one-third  of  the  approach,  but  not  to  the  extent  of  the  TV  group.  The  TV 
group  had  substantially  more  problems  with  the  flare,  believed  to  be  due  in  part  to  glide  path  deviations 
just  priot  to  flaring. 

Based  on  the  data  collected  during  this  study,  it  is  considered  that  the  two  computer-generated 
imagery  systems  will  provide  more  effective  training  for  KC-135  CCTS.  No  serious  deficiencies  were 
identified  in  the  Day  or  Night  systems  which  might  adversely  affect  the  quality  of  training. 
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APPENDIX  A:  INSTRUCTOR  PILOT  SIMULATOR  PREBRIEF  TO  SUBJECTS 


SIMULATOR  PRELIGHT  BRIEFING  INSTRUCTIONS 

The  formal  preflight  briefing  will  be  given  during  a half-hour 
period  before  the  first  simulator  hour  of  the  day.  There  may  be 
exceptions  to  this  due  to  scheduling  adjustments.  This  briefing  is 
designed  to  acquaint  the  subject  with  the  content  and  conduct  of  the 
training,  the  basics  of  flight  in  the  simulator,  and  the  specifics  of 
the  landing  task. 

Brief  the  items  in  the  order  they  occur  in  the  attachment. 
Spaces  have  been  left  between  many  of  the  items  for  the  team  members  to 
add  specific  information.  The  team  chief  will  conduct  a brief  seminar 
with  the  IP's  and  experimenter(s)  prior  to  the  first  simulator  day  to 
derive  the  specifics  of  the  task. 


1 Atch 

Briefing  Format 
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BRIEFING  FORMAT 


1.  CONOUCT  OF  TRAINING 

a.  Left  seat  training:  Right  seat  instructor  - unlike  copilot  training  at 
CCTS. 

b.  Multiple  repetitions  of  a landing  task  - specialized  training. 

c.  No  intent  to  teach  copilot  procedures  - SAC  procedures  eliminated. 

d.  Research  environment  - note-taking,  data  collection,  quiet, 
standardization. 

e.  Asking  questions  during  training  permissible  - includes  preflight 
brief,  flight  and  debrief  only. 

f.  Performance  measurement  during  training 

g.  No  questions  during  evaluation  - these  are  assessments  of 
quantity/quality  of  learning. 

h.  Strict  timing  control  - start  on  time,  stop  on  time,  always  be  in 
place  five  minutes  before  scheduled  start  time  for  all  phases  of  training. 

2.  CONTENT  OF  TRAINING 

a.  Open  the  syllabus  and  explain  it  to  the  subject 

DESCRIBE:  Content  of  each  period 
Length  of  period 
Periodic  data  collections 

QUESTIONS:  Answer  "what"  but  not  "why"  questions 

b.  Approximately  8 hours  of  simulator  training  and  evaluation 

DESCRIBE:  Simulator  environment  - quiet  except  for  engine  sound, 
front  and  side  visual  displays 
Long  sessions  - rest  periods  as  required 

QUESTIONS:  Do  not  answer  questions  concerning  rating  scales  on 
evaluations 

c.  Two  KC-135  evaluation  flights  at  Carswell  AFB,  TX 

DESCRIBE:  Similarity  of  task  to  be  flown  in  aircraft 
When  flights  in  A/C  will  be  flown 

QUESTIONS:  Do  not  answer 
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3.  BOEING  707-123  SIMULATOR 


a.  Brief  basic  cockpit  layout  using  chalkboard  and/or  wall  charts  in  the 
briefing  room. 

DESCRIBE:  Seating  arrangement  & adjustment 

Instrument  panel  - important  instruments,  crosscheck 

Collins  108  Flight  Director 

Brief  Specifics,  Fuel  flow,  RPM,  throttle  use 

SPECIFICS:  A/S  bug,  no  precession  ADI,  IVSI. 

b.  Describe  the  visual  display  f ield-of-view 

DESCRIBE:  Visual  system.  All  - front  and  left  window,  runway 
environment  independent  window  for  right  seat,  runway 
size 

NOVOVIEW:  Point  lite  source  nite  environment,  horizon  glow,  blind 
spot,  airdrome  layout,  apparent  depth. 

TV  CAMERA:  Model  board  w/camera  sweeping,  daytime,  blind  spot, 
airdrome  layout,  basic  ground  references,  altitude 
restriction 

DAY  CGI:  Computer  imagery,  day  color,  angular  shapes,  ground  & 
runway  texture,  apparent  depth,  r/w  layout 

4.  BASICS  OF  FLIGHT  IN  THE  SIMULATOR 

a.  Difference  between  jet  trainer  and  heavy  airplane 

. 

1.  Smaller  angles  of  bank  - 18°  bank  not  to  exceed  30°  in  turn 
to  final. 

2.  Power  to  control  pitch  - explain  "Pendulum"  effect. 

3.  Relatively  "sluggish"  controls  - move  pressure  than  T-38, 
slower  response. 

4.  Throttles  right  hand,  yoke  & trim  with  left  hand  - unlike 
fighter  type  flying,  throttle  alignment,  span  of  right  hand 

5.  Tendency  toward  using  rock/dutch  roll  due  to  using,  area  of 
momentum 

6.  Greater  lead  points,  patience  after  inputs. 
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7.  Trim  wheel  - noise,  electric  from  yoke 

8.  Fuel  Flow  - approximate  throttle  movement  to  obtain  FF  changes 

9.  Arm  rest 

b.  Performance  characteristics  of  the  simulator 

1.  Relate  simulator  characteristics  to  KC-135 

(a)  Fanjet  response  in  sim  different  than  A/C 

(b)  Motion  response  to  varying  inputs. 

(c)  Approximate  bank  and  pitch  angles 

(d)  Slightly  more  response  in  simulator  over  A/C 

5.  THE  LANDING  TASK 

a.  Diagram  the  landing  task  on  the  chalkboard  and  explain  the 
elements  of  the  task. 

Draw  graphic  depiction  on  board  and  highlight  the  segmentation 
of  the  total  task. 

Where  to  begin  (a/c  position  in  relation  to  r/w) 

b.  Table  brief  how  to  fly  the  landing  task.  *Brief  winds  during 
period  1 desired. 

1 . Control  movements 

2.  Specific  parameters  (altitude,  airspeed,  ground  track). 

3.  Visual  references  - location  of  r/w  aim  point  in  windscreen 

- ground  points 

- lead  points 

- peripheral  cues  in  left  window 

4.  Composite  crosscheck 

5.  Performance  standards  - altitude,  A/S,  bank,  tdzn  limits 

6.  Stress  VFR 

6.  QUESTIONS  FROM  SUBJECT 

a.  Document  questions  and  answers 
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APPENDIX  B STUDY  SYLLABUS 


PERIOD  1 


Fll  FAMILIARIZATION  (COCKPIT) 

1.  The  subject  will  be  familiar  with  the  following: 

a.  Strap-in  and  seat  adjustment 

b.  Basic  flight  instrumentation 

c.  Yoke  controls 

d.  Throttles 

e.  Trim  control 

f.  Engine  instruments 

F12  FAMILIARIZATION  (FLIGHT  CHARACTERISTICS) 

1.  The  subject  will  fly  the  simulator  and  become 

familiar  with  the  following: 

a.  Simulator  characteristics  during  straight 

and  level  flight,  turns  of  15°-30°  bank  to  headings 
based  on  ground  references,  shallow  descents,  low 
approaches.  No  landings. 

b.  Trim  control  during  flight 

2.  Special  Requirements: 

a.  Freeze  may  be  used. 

b.  Maximum  altitude  for  familiarization  is  1500'  AGL. 

Dll  DEMONSTRATION 

1.  The  instructor  will  demonstrate  the  landing  task 
one  time. 

LANDING  TASK  (PRACTICE) 

1.  The  subject  will  fly  the  landing  task  beginning 
from  the  primary  reset  (PR)  point  until  the  time 
expires. 


60  MIN 
5 min 


20  min 


5 min 


15  min 
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2.  Special  Requirement: 


► 


a.  Freeze  may  be  used. 

b.  No  wind. 

LT12  LANDING  TASK  (EVALUATION) 

1.  The  subject  will  fly  three  landing  tasks  beginning  at  the 
PR  point. 

2.  Special  Requirement: 

a.  Freeze  may  NOT  be  used. 

b.  No  wind. 

c.  Performance  measurement  data  will  be  taken  by  the  IP, 

TC,  and  Experimenter  (machine). 

d.  No  talking  while  performing  LT : critique  between 
tasks  permissible. 

PERIOD  2 

LT21  LANDING  TASK  (PRACTICE)  - NO  EVALS  1 

1.  The  subject  will  practice  the  landing  task 
beginning  from  the  PR  point  until  the  time  expires. 

2.  Special  Requirement: 

a.  Freeze  may  be  used. 

b.  No  wind  until  IP  calls  for  it.  Begin  turbulence  at 
1/2  notch. 

c.  Increase  turbulence  to  1 notch  on  IP  request. 

PERIOD  3 

LT31  LANDING  TASK  (PRACTICE/EVALUATION)  1 

1.  The  subject  will  practice  the  landing  task  beginning 
from  the  PR  point  until  the  time  expires. 

2.  Special  Requirements: 

a.  Freeze  may  be  used  during  performance  measurement. 
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15  min 


.5  hrs 


.5  hrs 


b.  Wind  will  be  introduced  and  varied  between  conditions  no 
later  than  the  beginning  of  this  period  with  1 notch  turb. 


c.  Performance  measurement  will  be  taken  on  three  landing 
tasks  (by  the  IP,  TC,  and  experimenter  machines) 

PERIOD  4 

LT41  LANDING  TASK  (PRACTICE)  1.5  hrs 

1.  The  subject  will  practice  the  landing  task  beginning 
from  the  PR  point  until  the  time  expires. 

2.  Special  Requirement: 

a.  Freeze  may  be  used. 

b.  Wind  will  be  varied  between  conditions  with  1 notch 
turb.  Turb  off  1/2  mile  on  final. 

c.  A demonstration  of  the  landing  task  may  be  flown  in 
the  last  30  minutes  if  the  subject  has  not  flown  at 
least  one  unaided  landing  task  by  this  time. 

PERIOD  5 

LT51  LANDING  TASK  (PRACTICE)  1.5  hrs 

1.  The  subject  will  practice  the  landing  task  under  varying 
wind  conditions  until  the  time  expires. 

2.  Turb  off  1/2  mile  on  final. 

PERIOD  6 


LT61  LANDING  TASK  (PRACTICE)  60  MIN 

15  min 

1.  The  subject  will  practice  the  landing  task  for  (Eliminate 

15  minutes.  the  15  min 

practice 

2.  The  subject  will  take  the  checkride,  regardless  if  already 

of  proficiency  attainment,  during  the  last  45  reached 

minutes.  profic- 

iency) 
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PERIOD  7 


LANDING  TASK  (CHECKRIDE) 


45  min 


1.  The  subject  will  fly  two  landing  tasks  with  no  performance 
measurement  with  wind  and  turb. 

2.  The  subject  will  fly  five  landing  tasks  with  performance 
measurement  by  the  IP,  TC,  CIP,  and  Experimenter  (machine). 

3.  Special  Requirements: 

a.  Freeze  may  not  be  used  during  the  five  measured  trials. 

b.  Wind  will  be  varied  between  conditions. 

c.  THE  CHECKRIDE  MAY  OCCUR  EARLIER,  BASED  ON  ACTUAL 
ATTAINMENT  OF  PROFICIENCY. 
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APPENDIX  C:  DATA  COLLECTION  FORMS  AND  PROCEDURES 


Several  forms  were  created  for  collecting  the  different  types  of  data.  Figure  Cl  shows  the 
questionnaire  used  to  document  subject  experience.  Figure  C2  shows  the  experimenter  log  maintained  at 
the  experimenter  station  to  record  the  length  of  each  landing  task  and  interim  critique.  Subject  and  IP 
protocols,  significant  remarks,  wind  conditions,  and  other  important  items  were  recorded  to  document  the 
history  of  each  simulator  period.  Figures  C3  and  C4  show  the  two  page  test  pilot  progress  record.  The  IP 
made  an  entry  after  each  simulator  period  summarizing  the  training  that  occurred  during  the  period  and 
any  problems  encountered  by  the  subject. 


FOR  RESEARCH  PURPOSES  ONLY 


STUDENT  BACKGROUND  SHEET 


AGE  25 


ASDS  4001 
class  m 


Flying  Background  (Pilot,  Nay,  etc.  - designate) 


C.-ISO 


Other  Special  Experiences  or  Qualifications: 


FOR  RESEARCH  PURPOSES  ONLY 


Figure  Cl.  Student  background  questionnaire. 
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Figure  C2.  Experimenter  log. 


36 


Figure  C3.  Student  progress  record  - page  1. 


DAY  2 


PERIOO  4 (1.5  hours)  (Weak  areas,  problems.  Instructor  comment s.  Describe 
progress. ) 

i - _ L i A . A— - - -T-L  _ _ 


rnnrA.  aar\  ulti 

. j riit  ucrkQU  ujKrr.  rr\ llc/v 

^>r  Kirt,  14  t nt.ru  - 

a LUIT  m Tin«a»Tunfj 

rucj 


PERIOD  5 (1.5  hours)  (Weak  areas,  problems.  Instructor  comments.  Describe 
progress. ) 

r — . v . _ . i „ i i_l a i j 


gcaaaw  hw;  T.«.nrr 
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Figure  CS.  Basic  data  card.  Figure  C6.  Revised  data  card 


Each  segment  was  considered  a composite  of  the  various  subelements.  A scoring  rule  was  imposed 
that  prohibited  any  segment  score  from  being  higher  than  the  lowest  score  of  its  subelements.  For  example, 
if  “Smoothness”  and  “Hold  Altitude”  were  scored  as  Good  on  segment  2 but  “Hold  Bank”  was  scored  as 
Fair,  the  highest  score  which  could  be  assigned  to  segment  2 was  Fair.  A lower  score  could  be  assigned  to 
this  segment  if  one  of  the  unwritten  criteria  was  violated,  such  as  unsatisfactory  rudder  control  during  the 
tum. 

The  five  segments  and  their  subelements  are  shown  below: 

1.  Base  leg:  The  subject  was  normally  given  the  simulator  or  aircraft  on  the  base  leg  after  the  IP  or 
AC  had  stabilized  the  critical  flight  parameters  of  airspeed,  altitude,  and  trim.  The  subject  was  scored  on  his 
ability  to  maintain  straight-and-leve!  flight  holding  airspeed  and  altitude  during  the  base  leg  prior  to 
beginning  his  tum  to  final.  On  the  Session  1 card  (Figure  CS),  the  rater  gave  a letter  grade  for  maintaining 

altitude  (“Hold  Alt ’’)  and  a letter  grade  for  the  overall  performance  on  the  segment.  On  the  Session  II 

card  (Figure  C6),  a letter  grade  was  also  given  for  maintaining  airspeed  (“Hold  A/S ”). 

2.  Turn  to  Final:  The  tum  to  final  started  with  the  decision  point  for  beginning  the  tum 
(“Roll-ln”).  The  rater  marked  the  roD-in  as  early  or  late  by  checking  the  appropriate  box.  If  the  roll-in  was 
started  at  the  correct  point,  a mark  was  made  through  the  solid  dot.  During  the  tum,  a letter  score  was 

given  for  bank  control  (“Hold  Bank ”),  maintaining  altitude  (“Hold  Alt ”),  and  (“Smoothness 

”)•  On  the  Session  II  card,  a letter  score  was  also  given  for  maintaining  airspeed  (“Hold  A/S ”). 

Upon  rolling  out,  the  rater  indicated  centerline  alignment  by  marking  whether  the  subject  undershot  (a 
mark  in  the  left  “roll-out"  box),  overshot  (a  mark  in  the  right  “roll-out”  box),  or  rolled  out  on  centerline  (a 
mark  through  the  solid  dot).  A letter  grade  was  also  assigned  for  the  overall  performance  during  the 
segment. 

3.  Final  Approach  Prior  to  Descent:  This  segment  was  the  straight-and-level  portion  of  the  approach 
following  roll-out  prior  to  the  begin  descent  point.  During  the  simulator  sessions  and  on  some  aircraft 
approaches,  this  period  was  too  brief  to  score  and  was  left  blank.  When  it  was  scored,  a letter  grade  was 

given  for  maintaining  altitude  (“Hold  Alt 1’),  centerline  alignment  (“Centerline  Align 1’),  and  an 

overall  segment  score. 

4.  Final  Approach  Glide  Path  This  segment  began  with  the  descent  decision  point  and  terminated 
just  prior  to  the  flare  decision  point.  The  decision  to  begin  the  descent  was  indicated  as  early  or  late  by  a 
mark  in  the  appropriate  box.  If  the  descent  was  started  at  the  correct  point,  the  rater  placed  a mark 
through  the  solid  dot.  During  the  descent,  information  was  collected  at  three  points  to  assess  glideslope  and 
centerline  alignment  error.  These  points  were  approximately  3.5  miles,  2.25  mies,  and  1 mile  from  the 
runway  threshold.  At  each  of  these  points,  the  rater  recorded  the  vertical  velocity  shown  on  the  vertical 
velocity  indicator  (WI),  and  the  glideslope  deviation  and  centerline  alignment  from  the  Collins  FD  108 
Flight  Director.  The  WI  reading  was  the  actual  reet  per  minute  shown  on  the  instrument.  The  glideslope 
deviation  was  shown  as  it  appeared  on  the  glideslope  indicator  (GSI)  on  the  Attitude  Director  Indicator 
(ADI)  with  the  dots  corresponding  exactly  with  the  instrument.  Therefore,  if  the  subject  was  one  dot  width 
low  on  glideslope,  a mark  was  made  through  the  upper  dot  on  the  card.  High  glideslopes  were  marked 
againsi  the  lower  dot.  If  the  proper  glideslope  was  being  maintained,  a mark  was  made  next  to  the 
horizontal  line.  Centerline  alignment  was  indicated  in  a similar  way,  with  the  degree  of  error  being 
displayed  in  relation  to  dot  widths  of  error  from  the  centerline.  However,  the  mark  showing  aircraft 
position  corresponded  directly  with  the  ground  track.  So  if  the  aircraft  was  to  the  right  of  centerline,  the 
mark  appeared  to  the  right  of  the  data  card.  An  overall  letter  score  was  also  assigned  to  performance  on  this 
segment. 

5.  Flare,  Landing  Attitude,  and  Touchdown:  This  segment  began  with  the  decision  to  flare  and 
ended  with  the  touchdown.  The  decision  to  flare  was  judged  as  early,  late,  or  correct  and  marked 
accordingly  by  the  rater.  The  subject’s  smoothness  and  technique  in  flaring  and  holding  the  correct  landing 

attitude  was  assigned  letter  grades  (“Smoothness “Technique ?).  The  touchdown  point  was 

marked  with  a cross  to  indicate  the  lateral  and  longitudinal  location  where  the  main  landing  gear  touched 
the  runway.  The  Session  1 card  had  a 1,000'  calibration  mark  to  show  the  optimum  touchdown  point 
1 ,000'  from  the  runway  threshold.  Two  additional  distances  (2,000'  and  3,000')  were  added  to  the  Session 
II  card  (Figure  C6).  An  overall  letter  score  was  also  assigned  to  this  segment. 
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APPENDIX  D:  ANALYSIS  OF  NONPROFICIENCY  SUBJ  ECTS 


The  following  tables  show  the  reasons  for  nonproficiency  of 
the  twelve  subjects  not  reaching  the  proficiency  criteria.  Each 
table  shows  the  segments  of  the  landing  task  on  the  left  margin 
and  the  appropriate  subject  number  across  the  top  margin. 
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